The “Cummins Experiment” on distributed software development yielded results that would be of interest to software companies, IT colleges and the students. The post builds upon our previous post on this subject and captures the gist of the detailed report of the experiment.
Acism can design software as an assembly of fine grained individual modules which are independent of one another. The size of a module is very small, so we expected that even a very junior programmer may be able to handle such a module. We went to colleges to explore whether we can work with their final year engineering students who are not in the job market yet.
The Head of IT Department of Cummins college, Pune, Ms Madhura Tokekar, liked the idea but suggested that we work with the 3rd year students who were about to get a month of vacation, and some of them might want to work with us during that time. Working with 3rd year students instead of the 4th year students made it even more challenging, but we went for it. We decided to work with all students who were willing to participate without any filtering. Since it was an experiment where we were not sure of the success, we kept the duration to be as low as one month. During that month, we would just send different modules to the students (for developing them from their college lab) and they would send those back after development — along with the unit tests to prove the successful working of the modules. The college spared a faculty to assist these students technically.
Acism side project leads would evaluate these modules. They were advised to be very stringent in their evaluations. If the students could push any modules through these criteria, then the experiment would be deemed successful.
We did not work with any real life project. We created modules to start building our own Kommbox. This kept us free from any delivery pressures.
We invited representatives from a few well-established software companies as industry observers – to monitor the experiment as it progresses and to guide in case it starts going astray. We are thankful to Persistent, Zensar, Synerzip and Mastek for nominating their key people as the industry representatives. That certainly encouraged us.
The experiment was performed over a month from Nov 23, 2014 to Dec 22, 2014. 10 teams comprising of 26 students participated in the experiment. Out of them, one team got dismantled in the initial phases and its members were absorbed into other teams. In the first week, the teams became confident with developing the page modules, even with the Twitter Bootstrap framework which they had not studied before. The JPA technology for interacting with database was a bigger learning curve, and week2 and week3 were almost entirely spent in learning. In week4 they started submitting modules based on this technology.
Apart from the steep learning curve, the module submissions got delayed due to a technical snag — from the college the modules could not be uploaded to our FTP server. In such a short timed experiment, even this kind of teething problems spelled a significant delay. In spite of all these troubles, everyone — the students, faculty advisor and Acism project leads — worked zealously to drive the experiment to success.
One attempt is one team attempting one module. The graph below shows how individual attempts transitioned from the steps: Started -> Dev done -> Bugs fixed (Ready) -> Submitted -> Completed. Completion could mean either acceptance or rejection of the module by the Acism project leads.
At the end of the experiment, 7 modules were accepted. More modules might have been accepted, if all ready modules were submitted. They were not submitted due to the technical snag mentioned above, and then towards the end we were rushed as the students’ academic classes were starting.
Here is the performance of the individual teams, which shows that barring the Eclipse team which was dismantled in the early phase, all other teams did a sufficiently good job in terms of completing their side development (which is indicated by the status of Ready or beyond).
Although the experiment was very short-lived as opposed to a real life software development project. It was achieving a steady state towards its end. The understanding of this steady state is important for us to extrapolate what one would expect if the duration were longer. In the below table, the two options are compared — with the same modular design of the software:
For this experiment, these outside resources were the students (whose efforts are marked with yellow background) and the core resources were Acism’s project leads (light blue background).
These were the observations for complexity 1 modules. For higher complexity modules, the %age requirement of inhouse resources goes down even further. For example, for complexity 2 modules, the inhouse resource utilization is only 16%.
The experiment was a success, and it proved that distributed software development can work in practice. Developers as junior as third year students can complete some modules meaningfully, and that goes towards a conclusive proof.
Conclusions of Interest to Software Companies:
Conclusions of Interest to Colleges and Students: