I recently gave a presentation on what it is like to work as a software developer to first-year engineering students at KTH taking an introductory programming course. I wanted to give my view on the main differences between professional software development and programming for a university course.
First I talked about challenges with large-scale software development. Then I listed several development practices used to cope with these challenges. I went on to present ways to become a better programmer, and ended with some fun facts from work.
Characteristics of Production Software
Programs are BIG. The main characteristic of software used in live systems is the size. For example, our main repository at work contains 1.8 million lines of Java (I used the tool cloc to find the number of lines). The sheer size of the programs complicates software development, and a lot of practices have been developed to deal with it.
Software is never done. Software that is used keeps growing and evolving. The customers find more and more uses for it, and want more and more features. Software that is not used is discontinued, but successful software is developed continuously for many years. Several developers are almost always involved. For example, I checked the subversion log of one of the main classes in our SMS application. There were 150 check-ins over 7 years, by 8 different developers (2 of which don’t work at Symsoft anymore).
Complexity from aggregation. Most features in production software are quite simple, but because there are so many of them, you get subtle (or not so subtle) interactions between them causing bugs. The complexity of the system comes from the aggregation of many simple parts, not from any complex parts.
Reading code. A consequence of the characteristics mentioned above is that reading code is a very important skill. Before a program can be modified, you need to understand what it does, and how it does it. Only then can new functionality be added so it fits in with the existing structure, and without breaking anything. Reading and understanding a program can be a major effort, and one sign of a well-designed program is that it is relatively straight-forward to modify it.
How to Manage
Many techniques have been developed to make developing and maintaining large programs easier. Here are the most important.
Modularize. This is an obvious first step, and (I believe) universally used. The software is split into subsystems, layers or modules so that smaller chunks of functionality can be dealt with at a time.
Iterate. Developing software bit by bit is as helpful for small 30-lines scripts as it is for systems with millions of lines of code. I like the following quote:
“A complex system that works is invariably found to have evolved from a simple system that worked.” – John Gall
Self-documenting code. The naming of the classes, methods and variables is incredibly important. The names, when chosen well, let you understand what the program does just by reading them. They also greatly reduce the need for comments in the code.
No duplication. Code-duplication causes a lot of problems when you come back to modify the program (happens all the time) – you may forget to make the intended change in all the duplicates. So instead of copy-pasting, combine the logic into one method. It makes the code more compact, and easier to modify in the future. There is an excellent article by Martin Fowler, Avoiding Repetition, on this subject. Unfortunately, I have seen a lot of duplicated code in production software over the years.
Unit testing. Unit testing (JUnit etc.) is useful both in small-scale and large-scale development. It’s an easy way to ensure your smallest program parts work as expected, and you get repeatable tests that can be run again and again. Making sure your code can be unit-tested also automatically improves the structure of the code – it becomes less monolithic, more de-coupled.
Version control. Using version control (like git or subversion) is a no-brainer, and as far as I can tell pretty much always used in professional software development. Version control systems are used both to keep track of different working versions of the software, and for knowing exactly what code is included in each release.
Version control is also useful on an individual level, for example for the programming assignments at a university course. Any program I spend more than 10 minutes writing, I stick in a local git repository. That way, I can always go back to previous (working) versions of my program.
Write for people first, computer second. The code you write will be read many times in the future (by you, or another developer). The computer doesn’t care how the code is written, so make it as easy as possible to understand for the next person that has to read it. A corollary to this is: don’t be too clever. It’s better to be clear than to be clever. The following quote puts it another way:
“It’s OK to figure out murder mysteries, but you shouldn’t need to figure out code. You should be able to read it.” – Steve McConnell
Plan for failure – logging and error handling. What do you do when the program doesn’t work as expected? You need some way of seeing what is going on. Usually this is in the form of tracing or logging. This will inevitably happen, so you might as well build in support for tracing and logging from the start (and make it possible to activate and deactivate while the program is running). It’s a similar story with error handling; put in place a unified way of handling errors in the program, because errors will happen.
Issue tracking. For any real system, it is necessary to have a way to keep track of bug reports – e-mails don’t cut it. At work we use Jira, but there are many products with similar capabilities. Often, the same system is also used to keep track of new features to be implemented.
Becoming a better programmer
There is almost no limit to what you can do to improve as a programmer. Here are some basic tips, appropriate if you are taking a software development course at university, but want to progress beyond learning the content of the course.
Program! The best way to learn to program is to actually program. Reading a book or listening to a lecture can make it seem like you have learnt and understood the concepts, but it is not until you actually start writing your own programs that you really learn. This is why (in my opinion) the best programming courses have lots of programming assignments – that’s when you are forced to put the theory into practice.
But you don’t have to stop at just doing the assignments. Solve some other problems with programs as well. If you don’t know what to solve, you can look for good problems at Programming Praxis, Ruby Quiz (the problems don’t all have to be solved in Ruby), or Project Euler (quite mathematical).
Learn a scripting language. Being able to write small scripts to automate tasks on the command line or to filter out information from a log file containing 100,000 lines is quite useful. Make sure to learn how to use regular expressions, and a language like Ruby, Python or Perl. All those languages are also used for “serious” applications, but even just using them for quick shell scripts is worthwhile.
Learn an IDE and a text editor well. For Java development it’s really worthwhile to learn to use an Integrated Development Environment (IDE), such as IntelliJ IDEA or Eclipse, really well (as I have written about before). In addition, it is always useful to know a good text editor, for example Emacs or Vim. In both cases (IDE and editor), learn as many keyboard shortcuts as you can. The goal is to be able to go from thought to program as effectively as possible.
Books. The number one book that every programmer should read is Code Complete by Steve McConnell. It’s always on top of every list of the best programming books, and for good reason (I’ve reviewed it on Amazon). It has a lot to say about how to write code (in a language-neutral way). The next book to read is The Pragmatic Programmer by Andrew Hunt and David Thomas. It contains a lot of tips on how to develop software efficiently.
Hacker News and Proggit. There are always interesting articles related to programming at both Hacker News and reddit/r/programming. Just remember to not spend too much time there – program instead. But for inspiration these sites are great.
Fun Facts
I also added a few fun facts about how we develop software at Symsoft. All of these are not necessarily true everywhere, but I wanted to add in a few bits of trivia to give a better sense of what it can be like to work.
HashMap and ArrayList. The two Java data structures we use the most in our code.
English. We’re in Sweden, but everything is in English – the code, documentation, bug reports etc. No big surprise, I just wanted to make sure everybody knows.
People interactions. Even if your job description is coding, there is quite a bit of interaction with other people. Mostly discussing designs and bugs with colleagues, but also contact with product management and sales, and with customers. There is no such thing as a solitary coder.
Time for bug-fixing. We schedule around 30% of the developers’ time for bug-fixing. This includes the time for investigating reported problems, fixing the bugs and testing the solution. It is worth noting that many reported problems aren’t actual bugs. Sometimes the system is not configured properly, sometimes the problem is in a surrounding system, and sometimes the bug report is actually a request for a new feature.
No UML. We don’t use UML, but there are lots of whiteboards around. We use them all the time when discussing a solution to a problem, and when describing how the system works.
The talk took about 45 minutes to deliver. It was interesting to prepare and fun to give. What is your opinion and experience on what it is like to work as a professional software developer? Let me know in the comments.
Henrik Warne's blog 
+1 for code complete and the pragmatic programmer. awesome resources and they are great to come back and read again every few years.
Good point about re-reading them! It is often good to be reminded about some of their tips. It was actually quite a while ago that I read The Pragmatic Programmers, so I already added into my pile of “to read” books.
Very good article! Totally agree with the reading aspect and better to be clear than clever. This is also elaborated in the book Clean Code by Robert C Martin (which is another reading tip)
“We are constantly reading old code as part of the effort to write new code.
Because this ratio is so high, we want the reading of code to be easy, even if it makes the writing harder. There is no escape from this logic. You cannot write code if you cannot read the surrounding code. The code you are trying to write today will be hard or easy to write depending on how hard or easy the surrounding code is to read. So if you want to go fast, if you want to get done quickly, if you want your code to be easy to write, make it easy to read.”
Exactly! Great quote! I haven’t read Clean Code, but I have read good things about it.
Can you elaborate on the No Uml point? What are the main reasons you found that you prefer not to use it? Do you use anything instead of UML?
My experience using UML in the past is that it took a lot of work to create and update the diagrams, and that they weren’t all that useful. They didn’t capture enough of the solution to be helpful – it was easier to use the code itself instead.
In some of the external documentation we use simple sequence diagrams, but no graphical documentation of the actual code.
I think pictures are very useful when designing and reasoning about the system, and we use the whiteboards a lot when discussing solutions. But these are ad-hoc drawings, and we don’t save them.
It seems like UML was developed to address a problem that was largely eliminated by Java and C#. From an understanding perspective, a fully developed set of interfaces and abstract classes is almost as useful as UML; the difference is that when you’ve developed those code artifacts you are actually developing the project, not just talking about it.
UML would make more sense in other engineering disciplines, but in Software Engineering we are literally building solutions out of our thoughts. There is no clear benefit to entering your thoughts into the computer once for design, and a second time for actual construction.
Well, it all depends. When one starts to take a grab on the problem, the business process diagrams (which is not UML), and the use case diagrams are useful, just as the domain diagram. (The use case diagrams give you a huge aid to design the authorization system.)
Now one has the business (requirement) analysis, so it’s time to design. Here the component diagram (giving the overall overview about the system, plus the communication protocols among the components) is that might be created. The really tricky processes (like e.g. handshake with an external stateful system) might be analyzed on activity, sequence, or communication diagrams, but it worth the effort only if the process is really complex. The user interface of some application might be caught as a state machine – the screens are the states, the transitions are the controls on the screen. Sometimes it makes sense to draw it.
Do not draw class diagrams ever. I guess that’s what gave you the bad experience. Class diagrams are not really for design – use them to document and check instead (discovering the cross dependencies is easy if one reengineers the code). So here we totally agree, UML class diagrams are not for designing.
In some cases, the user screens are also drawn first (again, not really a UML diagram).
Very nice article. You may want to add a couple of points about a) working in distributed teams and b) resolving customer issues when you are unable to simulate customer environments
Thanks! Both topics are worthy of their own blog posts I think!
I have worked in distributed teams in the past, but I much prefer a local team. Here at Symsoft we’re lucky to have all developers together on one floor – it makes collaboration so much easier.
Not being able to reproduce problems is always a big challenge. Having good logging and tracing facilities in your production environment can go a long way to help – you may be able to capture the problem in the customer system. But of course there are cases where that doesn’t help either.
I really liked your post. As a big advocate of unit testing, I was also pleased to see your observation that writing code that can be unit tested leads to writing better code — that’s a feature lots of people overlook.
Pingback: Working as a Software Developer | My Blog
Great article! From my experience as a software developer, this paints a very accurate picture of the job. Thank you.
Such a good article, can relate with almost everything!
Rarely do I agree with everything written in a blog. Although I’m past the stage of needing this advice (it’s internalized already), I can endorse all that is said entirely. Thanks for sharing.
How much documentation do you personally write? Along with a lack of unit tests and terrible git habits, a lack of code documentation is one of my developer guilts. Do you have any recommendations on quantity/quality of comments and documentation?
I write external documentation of the features I develop. As for the code, my philosophy is (as far as possible) “let the code be the documentation”. Well structured and well named code goes a long way. I write Java doc for APIs, but not for methods only used internally.
I do however add comments when explaining something that is not obvious from the code (usually answering the question “why” more than “how”).
Thats sick! Keep it up!
Sweet
Hi, I’m a beginner programmer in the industry (less than 2 years of professional experience). Your advices really help! However do you have any idea of how much time ratio I should put for programming vs learning from books/blogs? Cheers.
Thanks Alan! As long as you have the mindset to keep learning, the specifics don’t matter so much I think.
There are also great free university courses available these days from Coursera, Udacity, EdX etc.
This post was quite popular on Proggit, lots of comments there.
I like “No UML” very much, I have never used it, and I like to write in white boards and draw digrams
Interesting post Henrik you basically summed it up, and one of the most interesting points you mentioned is code readability which can be the cause of lot of communication, and development problems if no standard and clear code was written by the team. Moreover, unfortunately several points can be underestimated from many software development companies, at least here in my country, leading to various productivity and quality issues.
Pingback: Links for December 12th through December 15th
Indeed a worth reading article for all software developer. you pretty much sums it what all dev should think.
Great article. Concise too!
“Software that is not used is discontinued, but successful software is developed continuously for many years.”
A great point. Great software will almost always be great software as long as it’s tweaked as needed. Why let something that is 90% of the way there go to waste and start from scratch when a few lines of code here and there can make all the difference?
Role and responsibilities of software developer is not easy as one think off, you need to have good talent to handle projects in a good way as well as should have good communication skills so that you can interact easily with clients and to your team.
Hi
that’s a very good post you wrote.
Gretings from Rumunia
Great article! This is the kind of information that should be shared around the
web. Shame on Google for not positioning this post higher!
Come on over and talk over with my website . Thanks =)
Good points! Thanks for the article.
Pingback: What are the best websites a programmer should visit? – Quora Bites
Pingback: A Simple Plan | Dennis Mohr - Online
Pingback: Best-websites-a-programmer-should-visit – op07n
Pingback: Useful websites for programmers – article-world.ir
Pingback: Best-websites-a-programmer-should-visit – techpost