In-house development vs Start up development

Simple put - developing software which will be using strictly inside your company is just big waste of money. Why?

Image two companies, one wanted to improve their logistic procedures and another software company which wants to earn money on developing logistic software. Both companies hire 20 talented developers starting to build logistic solution. First company’s development team builds beautiful product which helps to reduce 10% of storage expenses for the company. Second company’s development team also builds pretty product and launches it, and since their market was much more that one company – they saved 10% storage expenses of each company which buys their software. If solution was good enough – they earn times more money that first team. Easy math shows that spending resources to second project is much more profitable that to the first. Now since it’s all about money, if you are going to invest money and have to options to invest whereas risks are equal but dividends are 1 to 100, where do you invest?

Another thing is that after the end of project in case in-house development we have 20 employees, which have nothing to do. And of course this burden will affect our decision to whether or not to implement another project (which might be even less valuable that first one).

So, what to do if you have non software business and want to create software product for your business? Explore options available on the market. If there is a product which suits 90% of your needs – buy it, and you will save huge amount of money J Only if there is no such product – start thinking about implementing it. If you truly believe that there is available market niche, then hire best people, create start up, funded by your company and start developing.  In case of success – it’s WIN-WIN situation.

PS This idea has some exceptions. It’s ERP and reporting frameworks. Usually every organization has unique set of data and unique set of ERP system configurations. So after creating some reporting solution or ERP module customization – it’s not possible to sell it worldwide, it won’t feat other organizations.

PPS Intentionally outsorcing is out of scope. Maybe I will talk about it latter :) 

To DB or not to DB (part 1).

 My previous position was Data analytic in corporation from fortune 100. Since our era is data centric, you might imagine that successful companies have strong position in that area.. Not at all. I can compare my job with stamp collector. It was real hunting for data, gathering them into one fold, matching them with what you already have.  For example I knew about 5 different sources for countries list (that lists did not match)- and every department used its own! Same with product hierarchy, customer hierarchy, etc. Another problem was that data may come to you system through 4-5 another system (so think about delays, forget about online data and consider possible changes to original).

There were even separate project just to promote single customer identifier so all part of companies would use same id across all business processes - and took about 1 year to implement that.

Root of that evil? Databases! Well not as such, but that easiness to install, create and maintain your own database.  Thanks to MS - cost ownership was really low and administration was so easy, that even manager could handle it :) I am not sure about licensing - but it seems we had something like unlimited. So every new project would not bother to reuse existing database, instead it's created own db and multiple all that mess.

Another possible root cause – is project oriented approach which is cultivated in modern corporations. Let’s say one project implemented product hierarchy for example for engineering department. And after 1 year of completion of that project sales department decides to use that hierarchy in their web site. But in order to this sales people want to filter some products based on couple of attributes, which does not exists in initial hierarchy. Easiest way that could possible work – is to duplicate database, create some additional table or tables with that attributes. Right thing – is to add this attributes to initial table. But team which implemented hierarchy for engineering is long gone, only one support guy is left, who is afraid to do such changes.  And the project management simply does not have additional resource or political power to do changes in source system, but management still want local success with new project.  So in one minute bang!  - you have data duplicated and modified in second system.

Example with additional columns is not complex. Sometime you not only need to add new data attributes but you need to modify existing data structures. Project oriented approach will simply ignore any efforts to maintain data quality and to avoid data duplication.

All data a priory should be considered as enterprise level, no such thing is department level.  Refactoring of data structures is crucial, or you will end up with hundreds of databases entangled with one another. 

Plug-in decision

  This one is about plug-ins. Last week we had enormous amount of issues with our production system. It is very popular trading platform which allows to install plug-ins into it. So our biggest customer has this system installed with about 7 plug-ins - and it crashed last week more that 15 times.  The issue is hardly reproducible, and you need to put all plug-ins and high load on the system, in order  to be able to repeat it. Worst thing is that it's impossible to debug it or identify where lock, crash or contention is happening. The system just freezes - nothing special in logs. That is - failure in one the plug-in is causing an issue in the whole system.

  So after some straight forward analyze you may come to the conclusion that plug-ins is evil, and decision whether  they should be implemented - is always no.  Well - I think there are some exceptions.

First of all - Open source. Plug-ins are ok in freeware and open source, because no one is expecting prefect performance  from freeware and all the more so from freeware with 10 additional plug-ins. Also plug-able architecture encourage developers to participate in the project more. And is it a point of open source movement?

  For proprietary software  it is not that clear. Trading system which I was talking about in the start of the post has no underlying SQL database. It's data is stored in .dat file and it’s impossible to  modify or even read them in usual way.  So plug-ins were only way provided by vendor to use data of that system (and what the crappy way it is!). Still, the system is one leading Forex platform by now. So that decision to have plug-ins might be the wrong one but overall solution was good enough. Here is my idea .When the market is young, and you physically can't produce all necessary features (in reasonable amount of time) that customers want to get -  you might consider to let them to do it (by plug-ins).  But of course you need to keep in mind that 10 or 20 plug-ins installed by one of your customer could kill your system, and probably it's you who will be blamed for it!

 PS Last week I completely migrate from Mozilla Firefox to Google Chrome. Seems like Mozilla's memory consumption became unpredictable, and I tend to think that their plug-ins are something to do this that.

It's interesting that variety of plug-ins in Firefox was that thing which persuade me to switch from IE to Firefox 4 years or something ago. And now my top priority is stability and performance.

 PPS My team is writing a lot of unit and integration tests, but just one erroneous plug-in with out test could ruin our quality-aimed approach.

Decisions and solution. Foreword.

Funny thing, words decision and solution are translating to the same word in Russian language ("reshenie"). In software sphere - word Solution is much more popular . Maybe because it's good from marketing perspective: when you are selling something it's better to be "complete and ultimate" Solution.  But anyway for every solution - there are millions of decisions. Most interesting, that even making wrong decisions you can have at the end product good enough to conquer the market. In my blog I am going explore decision making process around software development  in a very general sense. Also I am going to log some of my decisions and analyze them afterwards.