About Open Source

I have been using open source software since I started my first job around 10 years ago. By then the world was very different, Microsoft still dominated most of the software world (Windows, Visual Studio, .NET Framework, MS Office, SQL Server), and open source was still a small part of the whole ecosystem. Yet open source spread up like a virus over the last 10 years, and now even Microsoft is forced to embrace open source (well, it's also CEO related I guess).

To me, there are two major advantages using open sourced software:

1. free of charge

2. able to inspect, modify and recompile the sources.

The first advantage is especially important when I started the job in a start-up company, where the budget was a problem by then. And the second advantage becomes more obvious when the company becomes bigger, the budget for software is no longer a major problem but the complexity of solutions is. Being able to modify the source code ourselves, we could totally control the software.

Some Experiences

Here are some live examples of mine that show the why it's important to be able to view the source code.

1. shadowsocks

   A few months after setting up the proxy of shadowsocks, I would like to set up some special DNS <-> IP mapping. However, after I updated /etc/hosts or /etc/resolv.conf, it has no effect at all. At first I thought it's related to some sort of hardcoding in the source code. However, after I checked the source code, I found that the DNS initialization is actually done in one-off during application start up. So after I changed the DNS entries in /etc/hosts, I have to restart the shadowsocks application to make the changes effective!

   Some other proxies, e.g. Privoxy, the DNS change is done on the fly for each request, but it's not true for shadowsocks. (It's probably because shadowsocks is TCP layer but Privoxy is of the application layer.) There is no log and no documentation about this at all, without checking the source code it's really difficult to link up the two parts.

2. Selenium HTML report plugin for Jenkins

   One major problem of selenium plugin is, it makes an ambiguous status of unstable. When the execution of suite fails, sometimes it gives a status of error and sometimes (for some reasons) it just shows 'unstable'. But 'unstable' has special meaning in Jenkins actually, it's more like success with minor problems.

   So I really want the unstable to appear as failed, but there is no configuration and documentation for that. However, after download and read the source code for 5 minutes, I quickly found the relevant part. The plugin uses the values of numComandFailures and numCommandErrors for the differentiation. All I need to do to preprocess HTML result files to a foolish plugin. (A better way is, of course, to change the source code directly, yet the compilation and testing cycle for Jenkins plugin seems to be heavy for me.)

3. sikuli

   From the beginning, I want to use sikuli for automated functional testing of mobile devices. The whole idea and structure of sikuli are very promising. However, in order to use it for mobile devices, a lot of work needs to be done. It's (almost) impossible to wait for those features. Yet it's relatively simple to add the functions myself:

• add default timeout period since mobile device got larger time for screen switch.
• fix drag and drop problem
• added screen capture for every step for verification purpose

• move the mouse after each action, so that the mouse cursor would not affect the screen capture.

  If the project is not open sourced, I would probably have to abandon this as the original project could be hardly used for automatic testing purpose.

Contribution

After modified the open source projects for a long time, I really wanted to contribute back. A few months ago I finally joined the project of trac and started to develop on various minor tasks.

The main reason for selecting this project is that I have actually used Trac for more than five years, and I have already modified a lot of source code there, so I am more or less familiar with the source code. By getting involved in the project, I could get more familiar with this tool and develop future features more easily, meanwhile I could pay back to the community by the contribution.

I heard that many open source communities are harsh and closed, and don't welcome the new comers. Fortunately that's not the case for mine, at least the community for Trac development is very helpful and helped me solve a lot of technical problems when I try to set up a development environment. And they also advised me the tickets that I am capable of tackling with.

Here are some most important things I learned by participating the open source project:

1. Coding style. The coding style is much more important and strict than I thought. The community literally watched every single line of source code, and comments problems with coding style. In fact the code did look much better after fixing the coding style items. Eventually, I enforced pylint checking in my office as well.

2. Unit tests. For the contribution of code, I guess I spent 40% of the time for writing the code, and 60% of the time for writing the unit tests. Unit tests not only make the code for easier regression testing, and it actually made me try to better refactor the code. When the code is more modular, it would be easier for set up the unit tests.

3. Learn from others. When writing the new code to the project, and under the strict review process, I have no choice but reading a lot of other's work first before finalizing mine. You know the code review process is strict, so you would treat the whole thing very seriously, because you won't want to hand in some stupid code there.