Posted By : Dan Wilson     May 22, 2015 5:50 AM

Kickstarter is all the rage, isn't it? As the owner of 3 small businesses, I really like the capabilities kickstarter brings to the table. The ability to fund a project based on merit alone, rather than create and shop a business plan to the venture capital industry, makes a ton of sense. My only complaint is I didn't come up with kickstarter and make it my business before they did.

I've been working from home for 9 years. I find myself to be much more productive when I'm in control of my environment. Coffee is made the way I want, when I want it. I can turn my music up to help me blast through mundane tasks. I can have complete quiet when I need it.

I have had this nagging feeling over the last few years that my desk/chair setup isn't what I need. I have a very hard time maintaining correct posture in a chair. When I need to concentrate, I often sit in very uncomfortable postures that cause pain.

Yeah, so what does this have to do with Kickstarter?

My friend Dan Skaggs turned me on to a kickstarter project that is offering to make a high quality motorized standing desk. I decided it's just what I needed. What I need is a way to get a standing desk the EXACT height I need for a comfortable work environment, and also a way to return the desk to a sitting position when I want.

Also, if I decide standing desks aren't what I need, I can keep it at a sitting height. So no fear of commitment, right?

The project is over the original $50,000 goal by a long shot, so others feel the same way as I do. If everything goes well, I'll have a standing desk shipped to my house by July. It'll take a few weeks for me to get adjusted to working in a standing fashion, but I have some fairly reasonable hopes that if I stand for a portion of the day, I'll be able to solve my bad-posture-under-concentration problem to a reasonable degree.

The Project

Take a look at World's First Smart, Connected Office Desk -- Powered By AI.. They have a smart option, with a phone app, and a regular option without all the electronic whiz, bang capabilities. I chose the regular option because I didn't see the need for all the bells and whistles, when I'm just getting started. After reading more about the capabilities, I may be starting to regret my decision. I am now 50%/50% the smart option would be worth the money for my specific purposes.

I also ordered mine without the table top. I have an idea to make a very cool, custom wood top. For now, I can use glass desk top I have now.

Really, in a nutshell, I'm out $348 for a motorized standing desk (with no top) shipped to my house. That's a pretty good deal and is a lot more cost effective than the other standing desks out there that often cost over $1,000.

Once I get the desk, set it up and work through my initial growing pains, I'll post on my thoughts.

Posted By : Dan Wilson     May 18, 2015 6:11 AM

For reasons too complicated to get into, ColdFusion 11 no longer ships with a JDBC driver for MySQL. This is a change over previous versions of ColdFusion. Largely, this is because of a license issue relating to redistributing the Jar file.

How to fix it

The fix is easy:

1. download the JDBC driver JDBC Driver for MySQL (Connector/J) here: http://dev.mysql.com/downloads/connector/j/.
2. in the Platform select box, choose platform independent
3. next, choose the tar or zip version appropriate for your platform. If you don't know which to pick, choose the zip version.
4. next, on the screen titled "Begin Your Download - mysql-connector-java-5.1.35.zip", scroll to the bottom and choose the small link "No thanks, just start my download."
5. finally, once the download completes, extract "mysql-connector-java-5.1.35-bin.jar" and put it in the lib directory of your CF install. For me it is located at: C:\ColdFusion11\cfusion\lib (your platform and coldfusion edition impacts the path for you.
6. restart ColdFusion and validate your datasource in the ColdFusion Administrator.

That's all there is too it!

Posted By : Dan Wilson     May 14, 2015 12:38 PM

I've just posted the presentation "Our application got popular and now it breaks!" that was delivered at Dev.Objective() 15. If you were at the conference and would like the presentation files, you can find a PDF version here. You can also view this presentation on Slideshare

So you helped build a cool application and the public is flooding your site with traffic. What's that smell? Is a server on fire? Oh No! If only you'd thought more about the performance implications of some of your decisions, things would be better. Now, the phones ring at 4AM with angry customers, rousing you out of bed. There isn't enough coffee in the world to get you through these disastrous weeks ahead. You could have avoided all of this by coming to this session.

Bullet Points:

• Simple design decisions that come back to bite you
• How to use infrastructure for fun and profit
• Making the data tier work for you
• Performance traps and how to get out of them
• How to be sure your application can handle popularity
• The audience should have a working knowledge of server programming

Posted By : Dan Wilson     October 22, 2014 11:07 AM

At the CFSummit 2014, I presented on The Top 5 Things You are Doing Today to Hinder Scalability.

I collected my material through helping clients to scale their applications over a number of years. The important things in this presentation are listed in order. Decisions you make in your applications today, affect what options you have when you need to scale your application.

Certainly it is a very good thing to have an application you built grow to the point you need to consider scalability. Popularity is good, right?

However, there are decisions you can make in your code, code architecture and infrastructure architecture that will add or remove scalability options.

The presentation was well received by the audience and I thank each and every one of them for choosing to spend their time with me during this time slot.

For brevity, I included a number of details in an appendix to the presentation. Review this if you want to know particulars about a specific topic.

You can review the slide deck here: http://www.slideshare.net/ColdFusionConference/top5-scalabilityissues.

I'm always available for questions or consulting, should you need extra help.

I hope you enjoyed the CFSummit 2014. See you next year!

Posted By : Dan Wilson     July 23, 2013 9:14 AM

I'm helping prepare the CFSummit conference. We've organized the sessions on Trello and had a public voting session. It's time to start organizing the topics into a schedule.

In a conference schedule, it's important to know which sessions will be popular. It's desirable to ensure the most desirable sessions do not compete with each other. Thus, I wanted to pull out the sessions and organize the sessions by popularity.

The MongoDB Aggregation Framework

The MongoDB aggregation framework is a relatively new addition to the platform. Using this framework, you can group, sort, calculate and handle information gathering in the aggregate sense. Here's how I did this for the Trello Json data.

The Mongo Query

Exporting out of Trello gives a big JSON document with JSON members for each card. It turns out, in our case, all of the cards we want belong to a specific list. Once we pull the correct cards, we want to sort them by their votes. We'll end up with a sorted array of sessions by popularity. Here is the MongoDB query:

Explained Line by Line:

db.cfsummit.aggregate([

Notice the argument to the aggregate command is an array? This means you can organize a series of document transformations into steps. Each step will manipulate the document in some fashion. Let's look at our first step in the transformation:

{$project: { "cards": "$cards"}},

The first transformation is a $project command. Project (Pro-JECT), means to project a new way to view the data. In this case, I'm only interested in the cards node. The result of this document is a new document with basically only the cards member. You can write queries without $project, but I always do use it for 2 reasons. Firstly, reducing the size of the working document makes the query more efficient. The resulting projected document is smaller and can more easily be manipulated. The second reasons is I write my queries incrementally, so I only need to see, what I need to see. (Note the cards member is an array, this is important in the next step)

{$unwind: "$cards"},

Now the card nodes is an array. I'm going to want to sort all of the matching cards by the votes parameter. I use an $unwind command to transform the cards array members into their own documents.

Note, the cards member is no longer an array... this is important for grouping, which we will do later.

{$match: {"cards.idList": {"$in": ["51c9aa15d0b4871a3e000075"]}}}

Each of the cards we want to deal with belongs to listId: 51c9aa15d0b4871a3e000075. So we use the $match command to match the cards with the listId we are looking for. (Think of this like a where clause in SQL, if that is your background.

{$project: {"_id": 1, "name": "$cards.name", "members": "$cards.idMembers", "url": "$cards.url", "votes": "$cards.badges.votes"}},

Now I have my sorted cards belonging to the correct list. I now want to set up the return data structure in a way that is most useful to me. In my case, I want the ID, Name of the Session, The Presenters, The Trello URL for the card and the Votes Received. We once again use a $project command to organize the data in the format we want. Note, I've used a dot delimited path to walk the JSON tree to the data member I want. Hence, the votes were in the Votes Node which is inside the Badges Node which is inside the Cards node.

{$sort: {votes:-1}}

Lastly, we need to sort the cards by their popularity. The $sort command takes a JSON object containing the nodes you want to sort by. We want most votes to appear first, so we assign a -1 to the votes column for descending sort. Changing this to 1, would sort the data in an ascending manner.

Final Data Result

Want More Information?

Learn more about the MongoDB Aggregation Framework at their documentation site. You can install MongoDB in very little time and start working with data.

Free Training

If you want a more structured training, 10Gen offers a 7 week online training class on MongoDB for free. The classes are very well done. Consider a class if you are Mongo-Curious.

Posted By : Dan Wilson     July 21, 2012 10:12 AM

There are a lot of ways to cache data. You can cache a piece of data, a query, a page fragment, an entire page, or an entire website. You can cache to local memory, local file storage, distributed memory, distributed file storage, a front cache, or a Content Delivery Network (CDN). You can cache for ever, until the process regenerates, 5 years, 5 months, 5 days, 5 hours or 5 minutes. Heck, it might even, depending on the system, make sense to cache something for 5 seconds. Maybe less.

Why would I cache something for 5 seconds?

I know, I know, it seems silly to cache something for 5 seconds. You probably think this is a silly attempt at a ridiculous headline to grab clicks. However, let's explore. To get much benefit from caching, cache the content longer than the service time. The service time is the total amount of time it takes to service the request and return the desired item. As an example, if a piece of content takes 5 seconds to generate, the service time is 5 seconds. To get any real benefit, we should cache the content for longer than 5 seconds.

What happens if the service time is longer than the cache time?

If the service time is longer than the cache time, requests for the piece of content will queue. With caching, we want to AVOID queuing, so it's important to know the service time of the call under a variety of circumstances. You mathematical types can read up on Little's law, if you are curious: http://en.wikipedia.org/wiki/Little%27s_law

A practical example

Now, most cachable content has a service time of less than 5 seconds. Let's talk about what would happen in 2 identical systems. To make things simple, we'll pretend the following:

• There is only one process
• The service time of the process is 1 second
• The request levels are 1, 5, 15 and 30 requests per second.
• The non-cached system is real time, the cached system is cached for 5 seconds

In the non-cached system, here's how the requests per second (RPS) would look during 1 traffic hour:

• 3,600 @ 1 RPS
• 18,000 @ 5 RPS
• 54,000 @ 15 RPS
• 108,000 @ 30 RPS

WOW! I bet we'd have some major problems in a real time system under these conditions. Let's compare with the system using a 5 second cache:

• 720 @ 1 RPS
• 720 @ 5 RPS
• 720 @ 15 RPS
• 720 @ 30 RPS

Hmm, that looks odd. The requests in the 1 hour period never get over 720. Seems like an insurance policy against load.

Let's look at the amount of requests we save at each of the levels:

• 3,600-720=2,880 @ 1 RPS
• 18,000-720=17,280 @ 5 RPS
• 54,000-720=53,280 @ 15 RPS
• 108,000-720=107,280 @ 30 RPS

Wow! By caching for 5 seconds, we saved between 2,800 = 107,280 requests per hour.

What's more interesting we can see we established a service ceiling for our system. We'll never generate more than 720 requests an hour. No matter how many times the link goes viral on www.FunnyCats.com. As traffic rates increase, the value from a 5 second cache also increases. In a world full of email blasts, viral links, email, IM, social media, we see more and more bursts of traffic. As traffic bursts, we approach the natural threshold of a system. A system can only go as fast as the slowest part, (http://en.wikipedia.org/wiki/Amdahl%27s_law) so we need to make sure the slowest part is good enough for what business problem we are trying to solve.

So should we cache everything at 5 seconds?

A 5 second cache isn't the answer to every problem though. Some content can't be cached at all. Like a shopping cart. Some content can be cached forever, like static content (named with a version number). Some content types do not seem cachable, but maybe could possibly be. An example would be a page listing inventory. Perhaps the business is able to fulfill limited quantities of out of stock items. Maybe the products don't change often and come in all the time. In this case, it might be ok to cache inventory for 5 seconds and handle out-of-stock items by delaying shipment. The right answer depends on the business problem and the constraints on solutions. Which is a better problem to have, a down website, or a few out-of-stock orders to deal with?

Just for fun, let's look at the difference in caching for 5 seconds and caching for 5 minutes over 1 traffic hour:

• 720-20=700 @ 1 RPS
• 720-20=700 @ 5 RPS
• 720-20=700 @ 15 RPS
• 720-20=700 @ 30 RPS

The new service ceiling is 20 requests per hour. This is better than 720, for sure. However the savings aren't as dramatic as the no-cache to 5 second cache example. The reason why this is important is because we must balance the needs of the business with the needs of the system serving the business needs. Maybe the business can't afford to have information stale for 5 minutes, but 5 seconds is a reasonable level. You'd only have to have your system able to support 720 hourly requests, versus 108,000. You can get away with a lot more at the 720 RPH (requests per hour) level than you can at 108,000 RPH. I put together some charts showing the point of diminishing returns on longer cache intervals. The first chart shows the total number of requests in a 30 minute period at different traffic level times. Note the 5 minute period as the point in which there is no benefit visible on the chart.

The second chart shows the total number of requests in a 4 minute period at different traffic level times. Note again, the 5 minute period as the point in which there is no visible benefit, considering the starting point.

The answer is, as always, "It Depends"

The right time to cache a piece of content really depends on all the elements in the equation. Caching, even in non-intuitive ways, can be used to solve business problems within the available solution constraints.

Posted By : Dan Wilson     July 3, 2012 10:24 AM

I don't envy Technical Recruiters. They have tough job trying to place people in roles the recruiter himself doesn't understand. I don't blame them. The ever shifting sands of technology can be confusing. What other field completely changes landscapes every 3-5 years? Who can possibly keep up with just the new Javascript frameworks that came out this year, much less 30 years of technological nuances.

However, most programmers will share stories of a recruiter email going like this:

Recruiter: Hi Fred, I came across your resume and I see you would be a perfect fit for a great developer role in Java with an awesome company.

Fred: I don't even have Java on my resume and I know nothing about it!!!!!

As to why we get these emails, I can explain by simply translating:

Recruiter: Hi Fred, I came across your resumeYou came up for some reason in our Applicant Tracking System and I see you would be a perfect fit hope you would send me your resume so I can get my $10 bonus, or even go on an interview for this so I get a $50 bonus for a great developer role in Java with an awesome company Hey, I really need a commission this month. The economy sucks and my kid needs braces.

Most often, I'll help recruiters where I can, because at the end of the day, they are helping someone get a new job. Most people take new jobs because they either don't have one, or they want greener pastures. So either way, it is a win/win. However, today I got an email on LinkedIn by an especially useless recruiter looking for help. Here's the email, minus personal information:

Date: 7/03/2012 Subject: Fun with developing!
Hi, I hope you don't mind me reaching out to you, but I was hoping you could point me in the right direction...
I'm working with a bunch of awesome, dynamic companies in the area (Raleigh, Charlotte, Greensboro and beyond!) that are looking for developers and programmers in all kinds of technologies (Php, Java, .net, mobile, and Project Managers just to name a few).
I was wondering if you or someone you know might be interested in some new opportunities.
My contact info is below, feel free to forward my information! Thanks for your time, hope to hear from you soon!
Blahblahblah Someco.com

Let's try to translate this one, shall we?

Date: 7/03/2012 Subject: Fun with developing!
Hi,
I hope you don't mind me reaching out to you, but I was hoping you could point me in the right direction I have no idea how to do my job. I was selling used cars and it didn't work out...
I'm working with a bunch of awesome, dynamic companies in the area (Raleigh, Charlotte, Greensboro and beyond!) I have no current clients, but I see there are lots of ads on Monster and Careerbuilder. I'm going to pitch candidates at anything that moves just to see if I can get my foot in the door that are looking for developers and programmers in all kinds of technologies (Php, Java, .net, mobile, and Project Managers just to name a few) I don't have any specific job orders to recruit for, so any warm bodies will help. Ya can't sell a product, if there is no product to sell, am I right? I'm willing to do anything. Does your cousin know how to spell 'Computer', send him to me so I can put a resume in the system and get a bonus.
I was wondering if you or someone you know might be interested in some new opportunities Look, we are having a fire sale on job candidates this week. Either I get some new candidates, or I get fired. I really don't want to go back to the used car lot.
My contact info is below, feel free to forward my information! and even though I just sent you an unsolicited email, I want you to personally vouch for me, and spend your time forwarding my contact information to anyone with a pulse, because as we know, there is a severe shortage of technical recruiters out there. Thanks for your time, hope to hear from you soondon't forget, my kid needs braces so hurry up, willya?!
Blahblahblah Someco.com

What can we learn from this?

If you ask for help, please ask for specific help. Remember, the other party is going to weigh the benefits of helping you and if it isn't clear how you want to be helped, it isn't clear why you SHOULD be helped.

I'm not going to reply back to the recruiter, because I'm not really sure how to help them.

What sort of recruiter stories do you have?

Posted By : Dan Wilson     June 21, 2012 2:45 PM

Local Job Posting for friend. This is a good position for a high quality ColdFusion person who understands start up culture and wants to make a difference!

---

We are seeking a Senior Web Developer / Architect / Dev Ninja to join our team. Be a part of a young organization with a disruptive idea and good momentum in need of your technical creativity and expertise. The chosen candidate would have the opportunity to help establish the processes and standards for the team and contribute to high-level architectural decisions. We want someone who has "been there and done that" who can help us plan for the future and avoid common growing pains as the team and application increase in scale.

The goal of our company and application is to increase efficiencies in athletic organizations by leveraging the latest technologies to improve collaboration, communication and data analysis. Our application is used daily by coaches and athletes at several top universities and your contributions would have a direct impact of the workflows of these organizations. We are a small and informal but fast-paced team that likes to have fun while delivering impactful solutions.

Responsibilities:

• Collaborate on technical vision and direction for application and integration with other systems or devices.
• Produce code in a timely manner that is reliable, scalable and secure.
• Provide guidance on refining the entire SDLC including requirements, project management, source control and deployment.

Qualifications:

• 5+ years of development and architecture experience working on a large web application.
• Experience with object-oriented programming and methodologies including common design patterns with knowledge of ColdFusion or other java technology preferred.
• Must demonstrate ability or previous experience with scalability concepts and strategies such as caching at various levels in the stack.
• Experience with TDD and unit testing.
• Working knowledge of HTML, CSS and JavaScript a plus.
• Good knowledge of relational database systems and SQL would be preferable.
• Ability to write technical and functional specifications.
• Experience with Agile concepts and methodologies.

This is a full-time position and you would be expected to either live in the Triangle area or be close enough to spend most of your time at our office located in Durham, NC. Please send a resume and links to any previous projects if possible to jobs [at] lasllc [dot] com

Posted By : Dan Wilson     June 14, 2012 7:47 AM

Oracle Error - Oracle Not Available

I'm working on a client project that uses an Oracle 8i database. We'll eventually convert this database to another platform at some point, but for now, we need to make some much needed changes to the existing platform.

Oracle 8i doesn't seem to run on modern Windows Operating Systems so I installed it on Windows XP. This worked fine until I restarted the machine. Upon restart, the once functioning database service would not open. Connecting to the database gave the error "Oracle not available".

It turns out, this is a common issue and after researching and exploring various options, I finally got the database to start up with a series of steps.

Here's what to do:

Since this process involves starting services in a particular order, we need to change the 5 Oracle services below to start up manually: (Administrative Tools > Services )

1. OracleOraHome81TNSListener
2. OracleOraHome81DataGatherer
3. OracleOraHome81ClientCache
4. OracleOraHome81Agent
5. OracleWebAssistant0

While you are in there, change the name of your particular database service "OracleServiceWhateverYourServiceNameIs" to Manual also.

After a reboot, start all 5 services in the order listed above. Once all services are up and running, start your database service: "OracleServiceWhateverYourServiceNameIs"

It'll probably complain with an error afterwards, but that's ok.

Go to Task Manager and kill the ORACLE.exe process running and restart the service for your database instance: "OracleServiceWhateverYourServiceNameIs".

Try to connect to the database. Sometimes the service will be up and ready for service after these steps. If it is not, perform the following steps:

1. Open the Database Configuration Assistant ( start>Programs>Oracle-oraHome8i>database administration> database configuration assistant )
2. Choose "Change Database Configuration"
3. After pressing Next, choose the instance you want to connect to.
4. Press next 2 more times and the database will be ready for service then

At this point, you should be able to connect to your database with SQLPlus, or any other preconfigured connection. I hope this works as well for you as it worked for me.

Posted By : Dan Wilson     June 6, 2012 11:07 AM

I got a strange error "[Oracle JDBC Driver]Transliteration failed, reason: invalid UTF8 data" while working on a client system. I spent a reasonable amount of time trying to work out what caused this.

The Oracle database was a restore of an Oracle 8.1.6 system onto the new Oracle XE 11.2. During the import, the character sets changed.

• export client uses WE8ISO8859P1 character set (possible charset conversion)
• export server uses WE8ISO8859P1 NCHAR character set (possible ncharset conversion)
• import done in WE8MSWIN1252 character set and AL16UTF16 NCHAR character set
• import server uses AL32UTF8 character set (possible charset conversion)

So, I'm guessing since the new database did a conversion of NCHARSET from WE8ISO8859P1 to AL16UTF16, the size of the characters threw off something. Thus, there were problems and none of the queries on certain tables worked.

The Solution

The DataDirect Oracle Driver that ships with ColdFusion 9 has an error in it. It appears the error is fixed and if you have an agreement with the provider, you can download an update. However, I don't have an agreement so I downloaded fresh Oracle JDBC Drivers to fix the problem. Here's what I did:

• Download the drivers here: http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-112010-090769.html
• I used the ojdbc6.jar one.
• Copy the ojdbc6.jar file to /JRun4/lib (or, if you wanna be fancy, put it somewhere else and update the class path in the jvm.config pertaining to the instance you want to update)
• Restart ColdFusion
• Enter the following in the JDBC URL field: jdbc:oracle:thin:username/password@IP.Address.Of.Database.Server:PortOfDatabaseServer:OracleSID
• Enter the following in the Driver Class field: oracle.jdbc.driver.OracleDriver
• Add the user name and password in the appropriate boxes
• Save the datasource. It should verify if you did everything correctly.

If you got an error, remember these things:

• Usernames, passwords and seemingly the Oracle SID are case sensitive
• The JDBC Url Field is particular and must be exactly right.
• The default port for Oracle is 1521