MAKING A FINAL VENDOR SELECTION

The big day is at hand and the proposals will flow into your offices - some via overnight delivery, others by fax or e-mail. This is the first step to bringing your project to life as the vendors submit their most creative ideas, effective learning strategies, and step-by-step project plans.

Much preparation has gone into these proposals but if you are like most managers, you will skip the first 40 pages of each proposal and turn immediately to the price.

If your RFP was specific, the range in budget numbers should not be too great, perhaps plus or minus 30 percent. Now is your first chance to use the rule of thumb numbers given earlier in this chapter. If you were requesting two hours of high-end multimedia CD-ROM training on a topic of average complexity, you would expect to be looking at a price quote around $120,000.

If you have a vendor who comes in significantly lower than this ballpark number - say at $50,000 - you would want to look into why the number is number so low. Did they not understand the specifications? Do they have some unique cost advantage? Are they a garage shop with no overhead - and no infrastructure?

If another proposal is significantly higher - say $220,000 - you should similarly ask yourself why they are so high. Are they recommending a more complex or lengthier solution? Have they uncovered some unique problems the other vendors are not aware of? Are they really so good that they deserve a price premium?

After satisfying your initial price curiosity you need to carefully read the proposals to fully understand what you will be getting from each vendor. Pay attention to specific elements that can greatly influence price, among them, look at:

• Design strategies - linear versus simulations.
• Features such as book marking and score reporting.
• Amount of media to be used - audio/video and artwork.
• The number of animations and whether or not they are simple two-dimensional, or more complex 3D.
• Unique offerings such as instructional games.

Comparing Apples to Apples

You can not evaluate the prices you have received until you understand exactly what you would be getting from each vendor and how much it costs. A list of questions follows that you can ask vendors - in writing or on the phone - if their proposals do not give you the level of specificity you need to compare theirs with other proposals.

Answers to these questions will help you make true comparisons of RFPs:

• How much total learning time or how many screens will be provided?
• How much total audio and video time will be incorporated?
• Will professional or amateur actors and narrators be used?
• How many total graphics and animations will be used?
• What percentage of artwork will be original illustrations? What percentage will be stock photography or scanned photos?
• How many interactive exercises will be used throughout the training?
• What percentage of exercises will be simple multiple-choice questions? What percentage will be complex open-ended questions or simulations?
• How many questions will there be in the pre- and post-tests?
• What type of questions will be used in the tests (for example, multiple choice, true or false, matching, drag and drop)?
• What type of score reporting, tracking, or diploma printing will be provided?

Another price analysis strategy is to ask vendors to break out their pricing by different project tasks and to have them provide their hourly rates for each task. The most common tasks are:

• Instructional design and writing.
• Graphic art and illustration.
• Audio and video production - shooting and editing.
• Programming.
• Quality control.
• Project management.
• Out of pocket expenses.

This price breakdown will enable you to see if one vendor is planning on spending much more time on instructional design than another might be planning.

How to Evaluate Previous Work

After reviewing all proposals, you should be able to weed out some of the responses based on obviously inappropriate solutions or pricing. If you have not already met face-to-face with the vendors or seen their work first hand it is time to do so. Make sure the samples of work that are demonstrated are technically similar to the project at hand. If you want a low-bandwidth Web-based training program you will have little to learn from a vendor who is demonstrating the latest whiz-bang effects on a multimedia CD-ROM. If you have a large project you might want to ask the vendors that make your first cut to actually create a prototype or demo using your own materials. This is probably the best way to see the vendors' level of creativity and technical capability.

How to Check References

Checking references is a crucial step in the vendor selection process. Similar to a reference check for job candidates, this procedure requires that you talk directly to independent parties who have previous experience with the company. Any vendor that overstates his accomplishments or makes exaggerated claims will quickly be revealed through a few short calls to previous clients.

The key to effective reference checks is to request many references. Most RFPs only ask a company for three references, but every company has at least a few satisfied clients (or a brother-in-law who will claim to be a raving fan). You increase the odds of getting valuable client information if you ask for a minimum of five or six references.

When you call to check a reference, explain that you are considering working with Company X and would like to ask a few questions. You should start with general questions first, then probe for more detailed information. Questions can include:

• Can you tell me about the project they completed for you?
  After listening to the reference's answer you should examine how similar this previous project is to the one you will be doing. Consider whether the client's response is genuinely enthusiastic or cautiously subdued. Some references are reluctant to say anything specifically bad about another company but the tone of their reply will be a clue to their sincerity.
• Was the project delivered on CD-ROM or Web? Some technology-based training companies focus on only one type of delivery mechanism. Make sure the vendor has experience with the specific technology you plan to use.
• What time frame was the project completed in? If the vendor has completed a similarly-sized project in an equivalent time frame, then they probably also can handle your project's deadline.
• Was the project completed on time? If the vendor company is not large enough, it may not be able to complete your project in the given time period.
• How responsive was the company to making changes and revisions? How the reference answers this question will help you determine whether the vendor is customer-service driven. You can also ask whether the vendor tries to charge for every small change, or only for changes to the size and scope of the project.
• Was the project completed within the original budget or did the vendor ask for more money? Frequently, vendors will quote a low price to win a project, but then repeatedly return for more money throughout the project to cover "unanticipated" costs.
• What could the vendor have done better? No company is perfect but understanding a weakness in a previous project will help you avert potential problems should you choose that otherwise strong company as your vendor.
• Who was the project manager on the project? If you are dealing with a large vendor, you want to make sure you are getting references on the people who will actually be working on your project. The project manager, sometimes called the "producer," is the key individual who is responsible for quality, budget, and schedule. Try to get personal references on the project manager for your project.
• Would you work with the company again in the future? This question gives the reference you are calling the opportunity to offer the ultimate testimonial and will reveal just how satisfied their company was with the project overall.

How to Use a Vendor Score Card

If you have narrowed the possible vendors to one that is the most qualified and customer service oriented, you need only confirm standard contract details and your search is complete. However, you may find that there are a few vendors who all seem capable. In these cases a simple score card can be used to award each vendor points in various categories that you deem important. Figure 7.1 below provides a sample score card with a scale of 35 total possible points.