I recently got certified as Google Cloud Data Engineer and I wanted to share my path, motivations, and lessons learned as a software engineer without a data engineering background. I wanted to share why I think it is important to consider the value of generalism, why the software industry is getting increasingly wicked, and how we tend to walk arbitrary paths towards specialization.
The narrow path
It is not uncommon to feel discouraged for being a generalist. “A jack of all trades or master of none.” The mythical 10k hours to become a master; or any other ideal that might light the narrow path towards becoming an expert. This happened to me. Despite having a broad background, the circumstances brought me into front end development, where I spent quite some time trying my hardest to become an expert, following arbitrary roadmaps of topics and skills that I “should know” to be a true frontend master.
It’s hard to avoid noticing how this specific category of software tends to be caught as a prisoner of bias and how quickly it can discourage you to try different, “real” engineering topics. The industry gravitates towards these labels that end up pigeonholing people into subjective narrow paths.
Recently, I decided to pursue a Google Cloud Platform (GCP) data engineer certification to prove to myself that I am indeed a technologist. I decided this goal would help me avoid subjective categories, help me see what can I achieve or not, and get a first-hand understanding of topics that are outside my regular scope. With only a few computer intelligence courses on my back, I set out to achieve my goal.
I had in mind Epstein, describing Roger Federer and Tiger Woods’s path towards expertise, comparing the dynamics of sports like golf and tennis and how both excelled to the highest level despite focusing quite a different way during their upbringing. Being a single-digit handicap golfer, I am convinced that software development looks much more like tennis than golf.
Even when compared to a less constrained sport like tennis, software development is far from having absolute rules or finite solutions. Year over year, thousands of new ideas and frameworks arise. The buzz and hype around the next “big thing” revolve around us faster and faster, making it wicked to navigate through well-defined paths. But even so, deep down in the heart of the hustle and bustle of bleeding-edge tech, behind the fancy labels and cool buzzwords, familiar patterns and concepts of software per se lie unequivocally.
In the process of studying data processing systems and operationalizing Machine Learning models, I couldn’t help but realize that when seen at a bigger scale problems are more closely related than one would initially think. Principles such as:
- Ensuring flexibility and portability
- Ensuring reliability and fidelity
- Ensuring scalability and efficiency
- Designing for security and compliance
…fall into the common design, testing, and security patterns that, independent of the type of system you are working with, will and should be considered to ensure a quality solution.
A holistic approach
All in all, these detours from walking the broad path into deeper narrower paths are extremely enriching; being able to understand the dots even when the way to connect them is not yet clear. Integrating knowledge into a holistic understanding of software as a whole. This first step in getting certified left me feeling more empowered to take on diverse challenges, such as taking data-intensive applications to the cloud.
Some might argue that their specialized path is rock solid and they need not stray. But for the rest of us, those that don’t want to be put in a box, I encourage you to take a (free) wild peek into the adjacent paths. You might find some enriching contributions into your own software developer path, where the depth and width are constantly modified and built only by you.
If you’d like to hear more about my philosophies on being a generalist vs. specialist or learn how exactly I secured the GCP certification, you can contact me here. Let’s continue the conversation and share our different paths.