Why is software/website tendering so broken?
The way in which most organizations, especially government and non-profits commission software or websites is close to useless. Useless, massively expensive and wasting untold amounts of money. Does it really have to be this way? Are there rules or guidelines that enforce this kind of bad practice? Or is it simply a lack of understanding?
If these tenders were for architects, most of them would look something like this:
- We need a building
- It needs rooms
- It absolutely must have doors
- Windows are important too
- The front door should be extra large. It needs a keyhole exactly 143 cm above the floor. The lock must be made high quality steel and should have a blue edge painted around it.
- If possible, we would also like: An underground swimming pool, A helicopter pad, An amazing mural by an international artist
- Please provide a detailed project plan, with information on budgets and prospective timelines with your tender
- We won't provide you any information about what kind of budget we are thinking of
- Judging criteria will be 80% based on price
- You can ask one round of questions
- You have until next week to apply
- oh, and you must use agile (but that is only because we have been told we should do. We don't know what it is. We don't want to know what it is.)
- (and pages of drivel that has no relevance)
Can you see the problem? You get just what you ask for!
Software is complex. Developing a good website or software application requires lots of specialist knowledge, a lot of communication and collaboration. A tendering process that offers no opportunity for anyone to actually deliver something meaningful will not deliver anything meaningful.
For it to be possible to create good software, the commissioning process needs to clearly define the objective, problems that need to be solved, and the audience - the questions are far more important than the answers. The problem is not the technology, in most cases, the problem is finding the right problem.
When the underlying needs are clear, then those with the skills can work together to solve the problem. Unfortunately any tender application that specifies the solution it needs, when that solution was made up by someone with a poor knowledge of both the problem and the technology then you have nothing more than superstition. (Who you gonna call?)
How could we change the situation?
GDS did an amazing job creating the Gov.uk Service manual. It is probably the best non-technical resource on how to build agile projects on the web. I am certain it has had a major impact in bigger government projects. But, maybe it's still too big, requiring to much diligence?
Could we provide some simpler foundations that can help organizations write better tenders/specifications, or adopt a process that will not lead to so much waste?
The process of building software needs to be seen as a collaboration. Each party needs to see their work as a partnership, and should find ways to keep working within the realm of their expertise. When the developers are making up the original needs, or the person writing the spec is making up technological solutions, it's going to be a mess.
Are software companies to blame?
Programmers love not having to bother about actually creating tools that get used. It's much harder to build products that are meaningful, it's much more fun to just tinker away and fiddle with some new library and add another bit of swish. If an organization produced a detailed analysis of the problem they were trying to solve, would they be able to find a technology partner to work with them?
We can do better
I'd love to find a way to improve the situation. With some better education in this area we could save millions, if not billions. So where would you start?
What are those key ideas or skills or bits of process that should be clearly communicated for maximum impact?
And, even if the education is there, what are the elements of the 'system' that prevent better tendering and commissioning?
I've done a little more thinking on this topic. https://medium.com/the-organization/collaboration-the-weak-and-strong-forces-3b48927fdbc0
Peter, very important point you're making, the analogy is spot on. Without a discovery phase, any project of a significant size is doomed.
Seems to be client expectations, or people selling, "tech product X is easy, quick solution". To go with your analogy, when building a house, the builder already has plans, you often just get options to switch out a couple of choices. As the majority is already tried and speced, they can give you a time, it may be late but only for staff or weather factors. If you start with an architect like most web projects also do, you have experience of other buildings but you have never built this one before, though either the client things it is just like the building job, or sometimes it is sold to them like that. People who then become clients, need to at some time learn that they are asking for bespoke and have to price in learning and change, and not blame anyone for it. For this step with web I have tried Brennan Dunn 's roadmapping course as one way to approach discovery as a stage before specification. https://doubleyourfreelancing.com/roadmapping/?ref=tristanbailey If you can engage with the stakeholders and some research before you start the project will have a much better footing
One problem with many software tender requests is that they often provide little or no resource for the discovery phase. As with all good design this is an essential step, and may even be an ongoing process when trying to solve complex problems. I think the problem boils down to cost as most companies want a fixed price solution and a clear deadline; but large projects should really be delivered through an ongoing retainer relationship where roadmaps can be negotiated and milestones achieved.