This is the third part in a series on what it means to achieve mastery as a software engineer. The first part described senior software engineers, the second part discussed common flaws that, quite apart from coding skills, could undermine an engineer’s professional and personal growth. In this post, I’m going to try to get as concrete as possible about building basic skills – necessary but not sufficient first steps in moving from journeyman to master.

If you’re already an experienced developer, you might find this post to be a bit too basic, or you might disagree on one point or another. Please let me know if I’ve forgotten something important – I’d love to hear your comments! But if you look at the list and find yourself saying, “That’s not important! After all, I’m a senior software engineer and I never had to learn (or learned once and don’t remember) that,” then don’t be so sure. You may just have discovered a lacuna that’s been holding you back.

Note: Reading this post won’t make you smarter. It won’t make you a better programmer. It might not even be that entertaining (though I’ll try my best). It’s a syllabus that describes what you need to get from rank beginner to competent mid-level software engineer, and like all syllabi, is only going to be useful if you do the readings, go to the lectures, complete the homework, and study for the exams. Of course, this isn’t really a class, and the only test is going to be when you look back in five years and try to figure out how far you’ve come.

Basic Requirements

One of the things about mastery is that you don’t sweat the small stuff. Not because you have an army of interns, or because you’ve achieved a Zen state of equanimity and calm, but because you’ve gotten so comfortable with the basics of your craft that it’s become as natural as breathing. Everything in this category needs to be made so automatic that you don’t even notice you’re doing it any more.

• Languages

Lisp is worth learning for the profound enlightenment experience you will have when you finally get it; that experience will make you a better programmer for the rest of your days, even if you never actually use Lisp itself a lot.
– Eric Raymond, How to Become a Hacker

Learning new languages, especially those radically different from the ones you’re most comfortable with, will help you even when you aren’t programming in them. The more languages you learn, the more similarities you’ll notice between them, and the easier it will be to pick new ones up. The goal isn’t to become an expert in all of them, but rather to give yourself enough of a basis to understand the thought process behind them.

At the very least, you should know one of the following languages: C, C++, Java, C#. You may choose to use other languages for your projects, but these are languages that will teach you how to think about problems in a structured way. Prefer Lisp? Rails? Erlang? Great! Program in whatever language you want, but also know one of the imperative languages listed above. C and C++, in particular, will give you insight into how every other language actually works.

Whatever your language of choice, you need to be completely comfortable with it, from syntax to best practices. Your fingers should type in phrases, not individual keywords. I don’t know how many times I’ve asked someone which language they prefer, only to watch them write code that’s a third cousin once removed (at best). Don’t be that person.

Reference: The C Programming Language, Seven Languages in Seven Weeks

• Data structures and algorithms

I can’t say this enough – you aren’t a full-fledged software engineer unless you know the core data structures and algorithms backwards and forwards. If you don’t, you’re just a script kiddie assembling modules written by real coders. Experts have deep knowledge of their fundamental tools. You should know all about – and be able to code from scratch – arrays, linked lists, binary search trees, heaps, hashtables, sets, and graphs – including all their basic operations. You should know the different sorting and searching algorithms, DFS and BFS, and how to calculate minimum spanning trees, what big-O is, and the performance characteristics of different data structures.

It doesn’t matter what language you use, or what domain your problem is in. You may think that you can get away without using arrays (for instance), because Paul Graham said that lists were the way to go – but this ignores one of the most basic principles of CS, which is that big-O doesn’t care how fast your processors are, or how many you have – inefficient algorithms get expensive really quickly, and using the wrong data structure is one of the easiest ways to destroy performance.

Know your data structures. Know them really well. This one thing will make you a dramatically better programmer. Choose one per week and go through all its permutations. All it takes is reading a book and a little practice.

References: Introduction to Algorithms

• Logic / Problem-Solving

Some problems are hard, requiring deep thought about architecture, intricate algorithms, and mind-bendingly complicated data structures. However, the majority of the code you write, even in very exciting and difficult projects, is going to be fairly straightforward. There will be a feature to write, and you’ll have to understand the requirements, break the problem down into pieces, figure out the implementation details, and write the code. There will be some fairly standard questions to answer: recursion or iteration? Which data structure to use? Composition or inheritance? What are the edge cases, and how should they be handled?

“Being able to implement code that does what you want it to do” is the definitive characteristic of a competent coder. I don’t blame you for scoffing at the banality of the statement, because of course you can write code that does what you want it to do. And yet, based on over a thousand interviews in which I’ve given straightforward coding exercises to everyone from college students to senior architects, I can assure you that many people, even experienced software engineers, frequently can’t. Even when people can talk through a problem, they often have trouble writing it out in code. Sometimes they miss an important edge case, reverse a sign, segfault, or have an off-by-one error. Sometimes they blow it because they don’t understand a key programming concept. Sometimes their code is just five kinds of crazy.

Some people will never get there. Everyone else needs to write lots of code – put in their 10k hours – to build their skills. Code at work. Code at home. Ask your friends their favorite interview questions, and write out your own answers to them. Get code reviews from more senior engineers, and learn to avoid the anti-patterns and rookie mistakes that lead to brittle code. You can try reading puzzle books, or doing coding competitions, but ultimately, this is mostly about writing code, and lots of it.

Reference: Effective (Java|C++|Javascript|*)

Specifics

Though I’ve focused on general skills in the previous section, there are certain specific skills that all engineers simply need to have. These are skills so frequently useful, or essential, that I’m going to use a separate section to call them out.

• Unit tests

Know how to write unit tests. Try using several different frameworks. Understand what Test-Driven Development (TDD) is all about. It’s not sexy, it isn’t cool, but it’s an essential skill. ‘Nuff said.

References: Test Driven Development: By Example, Practical Unit Testing with TestNG and Mockito

• SQL

Being able to query and manipulate data in a traditional relational database is an absolutely essential, baseline skill. I’m not talking about designing complicated schemata, setting up geographically diverse multi-master configurations, writing MapReduce jobs, or NoSQL expertise. You simply need to be able to use SQL to extract data from MySQL, Postgres, Oracle, sqlite, whatever. Learn how to use inner and outer joins, sub-selects, primary keys, and the “explain” command. Understand the various data types, and get up close and personal with the most common math, grouping, and string- and date-manipulation functions. You just need to know this stuff.

Reference: I learned SQL on the job from Oracle manuals, many, many years ago, at my first internship. As such, I don’t have personal experience with a good learning text, but O’Reilly books are frequently a good place to start.

• Linux

OK, here’s where all the Windows and .NET developers are going to get angry, but yeah, you just need to know how to use Linux. Not at a deep, kernel patch contributor level, but well enough to use it – from the command line – as a primary OS. Linux-based systems show up all over the place, and you need to be able to interact with them, if only at a basic level.

Important commands are ls, cd, mkdir, rm, cp, mv, ln, cat, more, less, tail, grep, su, make, which, find, tar, pwd, exit, crontab, date, ssh, scp, ps, kill, echo, and vi (please, please, let’s not get into a vi vs. emacs debate – yes, emacs is extremely configurable, yes, you can extend it with Elisp, I know, I know, you’re very clever and I’m very impressed – but the point is that you need to be able to edit files on any machine, from a terminal, without having to install any additional software – hence vi); how to pipe and redirect output; how to install packages on your specific distro; how to set up your environment, and so on. The key here is that you should be comfortable working from the command line, not that you’re going to get so good at it that you’ll be able to chuck your IDE and write your code using ed.

Reference: A Practical Guide to Linux Commands, Editors, and Shell Programming (3rd Edition), Advanced Bash-Scripting Guide

• Object-Oriented Programming

OOP has been a dominant paradigm for decades. Even if you prefer a different model, understanding OOP, its benefits and drawbacks, and being able to use it effectively is an important basic skill.

Reference: Design Patterns

• Common formats and standards
  

There are a lot of common standards, and though to some extent this will differ by industry, there are some that are universal enough that you should just know them. ASCII, Unicode, XML, CSV, and a host of others so ubiquitous I can’t even think of them (for the same reason, as Neil Gaiman and Terry Pratchett said, that someone in Trafalgar Square can’t see England). Know what TCP and UDP are, and what an IPv4 address looks like. Let’s not get too crazy here – you need to know what these are, be able to use them (and sight read, in the case of text-based file formats), but unless specifically required for a project, you don’t need to get too deep into the weeds in any of them (at this level, at least).

Reference: only about a billion websites

• Text Manipulation

There are lots of ways to manipulate text without resorting to code, and it’s astonishing how often this turns out to be a useful skill. Whether you’re using Perl, Python, or bash, you should know how to use regular expressions, how to grab data from files, massage it, and output it in a useful way. For instance, one very common task is to search through a log file for lines with error messages, grab the part of those lines which identify which feature was involved, count how many errors each feature had, and output the features in sorted order – most errors to least. This can be done with a fairly complicated high level program, or a one-liner in Unix. Know your tools.

Reference: Introduction to text manipulation on UNIX-based systems

Best Practices

Experience is a dear teacher.
– Benjamin Franklin

Learning from experience is the worst possible way to learn something. Learning from experience is one up from remembering. That’s not great. The best way to learn something is when someone else figures it out and tells you: “Don’t go in that swamp. There are alligators in there.”
– Clay Shirky, A Group Is Its Own Worst Enemy

There’s been a lot of thought put into the problem of software engineering over the last 60 years. Some things change with new technologies, programming languages, and tools, but the most profound insights have generally been researched, tested, reflected on, well understood for decades, and transcend any specific technology. Do yourself a favor and learn them from a book instead of painstakingly trying to discover them on your own.

References: The Pragmatic Programmer, Code Complete (yeah, it’s super-long, but it will save you time in the end)

What’s Next?

If you’ve made it this far, you’re probably either an inexperienced developer trying to learn how to level-up, a senior engineer thinking about how to mentor a junior colleague, or one of my direct reports who knows I’ll ask questions about it in our one-on-one later this week (ah, power). Seriously, this is all very basic stuff, very foundational, and as such stuff that you probably already know, at least to some degree. Though there are some things you can just learn and be done with (the basics of data formats, for instance), many of these are life-long studies, things you’ll continue to get better at throughout your career. When you’re just starting out, these are on the critical path, because they’re your own personal minimum feature set for being an effective programmer.

In the next part of the series, we’ll talk about what happens when the basic skills are solid, and you’re looking to take the next step.