Distributed Software Development – The Cummins Experiment

The “Cummins Experiment” on distributed software development yielded results that would be of interest to software companies, IT colleges and the students. The post builds upon our previous post on this subject and captures the gist of the detailed report of the experiment.

Experiment Design

Acism can design software as an assembly of fine grained individual modules which are independent of one another. The size of a module is very small, so we expected that even a very junior programmer may be able to handle such a module. We went to colleges to explore whether we can work with their final year engineering students who are not in the job market yet.

The Head of IT Department of Cummins college, Pune, Ms Madhura Tokekar, liked the idea but suggested that we work with the 3rd year students who were about to get a month of vacation, and some of them might want to work with us during that time. Working with 3rd year students instead of the 4th year students made it even more challenging, but we went for it. We decided to work with all students who were willing to participate without any filtering. Since it was an experiment where we were not sure of the success, we kept the duration to be as low as one month. During that month, we would just send different modules to the students (for developing them from their college lab) and they would send those back after development — along with the unit tests to prove the successful working of the modules. The college spared a faculty to assist these students technically.

Acism side project leads would evaluate these modules. They were advised to be very stringent in their evaluations. If the students could push any modules through these criteria, then the experiment would be deemed successful.

We did not work with any real life project. We created modules to start building our own Kommbox. This kept us free from any delivery pressures.

We invited representatives from a few well-established software companies as industry observers – to monitor the experiment as it progresses and to guide in case it starts going astray. We are thankful to Persistent, Zensar, Synerzip and Mastek for nominating their key people as the industry representatives. That certainly encouraged us.

Experiment Progress & Observations

The experiment was performed over a month from Nov 23, 2014 to Dec 22, 2014. 10 teams comprising of 26 students participated in the experiment. Out of them, one team got dismantled in the initial phases and its members were absorbed into other teams. In the first week, the teams became confident with developing the page modules, even with the Twitter Bootstrap framework which they had not studied before. The JPA technology for interacting with database was a bigger learning curve, and week2 and week3 were almost entirely spent in learning. In week4 they started submitting modules based on this technology.

Apart from the steep learning curve, the module submissions got delayed due to a technical snag — from the college the modules could not be uploaded to our FTP server. In such a short timed experiment, even this kind of teething problems spelled a significant delay. In spite of all these troubles, everyone — the students, faculty advisor and Acism project leads — worked zealously to drive the experiment to success.

One attempt is one team attempting one module. The graph below shows how individual attempts transitioned from the steps: Started -> Dev done -> Bugs fixed (Ready) -> Submitted -> Completed. Completion could mean either acceptance or rejection of the module by the Acism project leads.

At the end of the experiment, 7 modules were accepted. More modules might have been accepted, if all ready modules were submitted. They were not submitted due to the technical snag mentioned above, and then towards the end we were rushed as the students’ academic classes were starting.

Here is the performance of the individual teams, which shows that barring the Eclipse team which was dismantled in the early phase, all other teams did a sufficiently good job in terms of completing their side development (which is indicated by the status of Ready or beyond).

Although the experiment was very short-lived as opposed to a real life software development project. It was achieving a steady state towards its end. The understanding of this steady state is important for us to extrapolate what one would expect if the duration were longer. In the below table, the two options are compared — with the same modular design of the software:

• Inhouse Development: The usual model of development in which the software is developed by company employees. Each developer has the entire codebase and goes developing one module after the other.
• Distributed Development: The core team does not do the actual development. It creates modules and evaluates them after completion. The work is outsourced either to out-of-company resources, or resources outside of the core team.

For this experiment, these outside resources were the students (whose efforts are marked with yellow background) and the core resources were Acism’s project leads (light blue background).

These were the observations for complexity 1 modules. For higher complexity modules, the %age requirement of inhouse resources goes down even further. For example, for complexity 2 modules, the inhouse resource utilization is only 16%.

Conclusions

The experiment was a success, and it proved that distributed software development can work in practice. Developers as junior as third year students can complete some modules meaningfully, and that goes towards a conclusive proof.

Conclusions of Interest to Software Companies:

1. There is a potential to bring down the use of key team members to 20% by going for distributed software development. In other words, the scalability increases to 500%. The actual development is outsourced to non-key resources (which have more availability) or even outside the company.
2. Junior resources can be employed to bring down the cost of software development, as long as their deliveries are thoroughly checked before accepting.
3. Even in the outsourced mode the IP is protected. The outsourcer has access to only the given module. The rest of the code is protected.
4. A hybrid approach is also possible where some modules are developed by key resources and some are outsourced.

Conclusions of Interest to Colleges and Students:

1. Module development can be used as a learning vehicle. A student may focus only on one module at a time, learn related technology and implement it practically on the module.
2. The students get a taste of real life development which is invaluable to form a practical perspective for their further academics.
3. The enhanced learning and the confidence to tackle technologies not covered in syllabus make the students great candidates for placements –especially in a market full of graduates without practical experience.
4. The practical work can help a college find gaps in students’ education and improvise their teaching and/or syllabus. This helps the colleges get better and better on a continuous basis.