This blog is for non-technical people looking to purchase custom software. Purchasing software is very difficult and very expensive. It is a decision that is hard to go back on, and can be the difference between success or failure of a product. Software development is a relatively young industry, and the benchmarks for judging software talent are not yet standardized. In my interviews with non-technical owners, entrepreneurs, and managers, I have heard the criteria is ambiguous. After talking with a few software providers whom I value about this problem, I have constructed the following lists as guides for how I would go through the selection process with software providers, agencies, consultancies, freelancers, or independent contractors.

Top 5 Things To Ask Potential Software Agencies

1. Do you write tests?

   This one is the biggest deal breaker, and the biggest qualification of professionalism. If they say no to tests, run away. Robert Martin compares agencies that don't run tests to doctors who don't wash their hands. Tests are what proves your system works and allows you to constantly change it without fear of breaking something else. In a new product lack of tests can stifle innovation as opposed to having the confidence to believe the sky is the limit. The software provider should also do continuous integration, making sure those tests are actually exhaustive and run often.

2. Ask what they do and don't do well

   Software is very expensive and complex to make. You want a partner who can advise you honestly to make the difficult choices. The best proof of their ability to advise is their ability to guide you in the selection process. Will they be making sure you are the right fit, or will they simply have money signs in their eyes? Most good software providers I have met are opinionated and know what they are good at and what they are not. Be wary of the provider that will do everything, as they will learn how to do it on your dime. Being comfortable with your limitations is a sign of a confident professional.

3. Independent code audit

   I recommend using Corey Haines or Sarah Gray from Technical Advocates. They can provide you with an assessment of the providers' technical competency. Because they are high-quality developers and entrepreneurs themselves, they are excellent judges of technical process and skill. It is worth spending a little money up-front to make sure your provider is highly skilled. If not, you risk spending an order of magnitude more to budget for a junky product. Unfortunately, that is a real possibility in our industry.

4. Community contributions

   Most products are built on top of open source software. Seeing someone's contributions tells you they have enough expertise that they see a piece of software and think they can do it better—a great quality to have when you are trying to build a new product. Their ability to be creative and innovative in their craft will create a common understanding of what innovations you envision in your product.

5. References

   Doug Bradbury taught me a style of interviewing potential hires that is based around the principle that past performance is the biggest indicator of future performance. How a software provider worked on previous projects will be the best indicator of how they will collaborate and deliver for you. Make sure the references are a combination of current and former clients. If they have clients listed on their website, try reaching out to them, too.

Top 5 Things To Remember When Negotiating The Contract

1. Get to know the team

   Ask to meet more than the sales person and management. Ask who will be on the production team, and to meet them before committing. It will ultimately be them whom you will work with everyday. Make sure you can build a rapport with them and trust them to deliver your project.

2. Know their subcontracting situation

   Small providers often need a little help when taking on multiple projects. Good providers have some trusted partners whom they collaborate within these situations. Ask for a list of all the companies they work with, and maintain a right of refusal if you don't feel the quality meets your expectations.

3. You own the IP and the source code

   You are hiring experts to develop your idea. All artifacts are yours and yours alone. Be wary of deviation from this by yourself or the provider. Make sure it is in writing that all vendors used by the software provider will be owned by your company. This means hosting, source code, error tracking, etc. Source code is the most important, for which I recommend GitHub.

4. No big commitments

   Make them prove their work incrementally. You don't want to get shackled in to one vendor for the duration of your product. A software provider who is confident in their ability to deliver will give you a weekly opt-out clause. If you aren't happy with the work they are doing, find another vendor. The ability to exercise the opt-out is important, but equally important is the pressure it puts on trust and collaboration. Build the relationship, don't buy it.

5. Contract reflects the relationship

   There is a large continuum of potential compensation plans, from paying for time and materials to signing fixed-bid contracts. Make sure the compensation plan fits your values, whether you desire the flexibility of unscheduled releases or reductions in scope through a time-and-materials payment method, or prepaying for risk with a fixed-bid system. The goal is for everyone to have a stake in the success of building the software product, and for both sides to be taking similar risks in the building process. An example is how we do the fixed feature model.

Top 5 Things Once You Engage With The Provider

1. Don't tolerate bugs

   Bugs are a symptom of the internal quality of the code. If things are incorrect in the demo meeting it often shows a lack of thoughtfulness and thoroughness. If old features break with new features being added, it is a sign of lack of testing and decoupling the system together. You need to nip these in the bud or they will be a multiple more expensive to fix later on. Robert Martin has a heuristic for this, "the only way to go fast is to go well." I am telling you what you know as an entrepreneur. Consistent mistakes are not acceptable. Bugs are not part of the process, they are something to be embarrassed about and quickly fixed without pay.

2. Plan releases

   Track the scope carefully between production deployments. Managing increases in scope is a collaborative process and can be very difficult. As you learn more about the product, you will inevitably need to make changes in scope. It is important to make sure there is an up-to-date plan so everyone understands the current scope and projected release date. The plan is a moving target that is worth the effort to update consistently.

3. Deploy weekly

   A software provider should agree to meet with you weekly to show your product's progress and functionality, and to make sure that it's the software you had envisioned. If it's not, the provider should be able to create new stories to pivot for the following week. This minimizes risk on both ends by making sure you are both on the same page throughout production

4. Plan in vertical slices

   A vertical slice is a feature that goes all the way through your system. For example, you would never have a reset password screen styled but non-functional. It is important as a stakeholder that you have the ability to release your product at any given moment. If something is "half done," it is debt against your system. To deploy anything, you have to finish the feature first. It is a liability to your system's ability to move forward and pivot with change. Make sure there is never any feature in your staging or production environments you don't want customers to see.

5. Only pay for what goes live

   A feature is not complete until it is integrated into your application. Make sure the product is in a deployable state at your iteration planning meeting each week. You should never be seeing a demo off of a developer's computer. Part of the development process is that the feature can go live at any time without breaking anything else. Don't accept anything less than that.