Hey there, pull up a chair, and let’s have a real heart-to-heart about what we all know has been happening this past decade in the world of DBAs. Yeah, I know, not exactly what you were looking for in my mostly technical blog, but stick with me here because this talk we’re going to have… Is very, very important.
You see, in this past decade or so, we’ve all noticed a trend where more and more businesses are looking for DBAs who know a bit about everything: RDBMS, NoSQL, Key-Value databases, Cache Databases, Full-Text Search Databases, Vector Databases, and more. In response, more and more data professionals prefer to become multi-disciplinary, doing a bit of everything… Even if it means becoming a part-time Data Engineer / Data Analyst / Data Scientist / DevOps / whatever. Otherwise, they fear that they won’t be able to find enough job opportunities.
But, there’s a cost to that approach. Sometimes a very high cost.
The Rise of the “Jack of All Trades”:
Lots of companies want DBAs who can handle different types of data systems. These “jacks of all trades” are pretty handy to have around. They can switch between different systems easily, which is great for businesses with lots of different types of data platforms… And it’s much cheaper for the business than hiring an expert for each data platform full-time.
The Value of “In-Depth” Expertise:
But even though being a jack of all trades is handy, sometimes you need someone who really knows their stuff in one specific platform. These experts bring deep knowledge and insights that can be crucial for complex projects. When the stakes are high, having an expert on hand can make a huge difference.
You Can’t Have Both:
The sad truth of the matter is that you can’t have both.
It’s physically impossible for one person to be both a “jack of all trades” as well as a “master of all”.
As the saying goes: “jack of all trades, master of none“…
On the other end of the spectrum, hiring a separate specialist for each data platform is going to be extremely costly for such a business.
Sometimes, if you’re lucky, there’s a compromised approach when a person is a “jack of all trades, master of ONE”, or even “jack of all trades, master of SOME”. i.e. they know really well one data platform (or some), and also just a little bit about many other platforms. Still, it’s not a perfectly ideal situation either, since there will still need to be a compromise somewhere. I mean, you don’t honestly believe the word “master” in this context really means “master” in the full sense of the word, right? There’s only so much one can fit into their brain.
Spiraling Down
On February 5, Adrian Nayrat published a blog post about the fear of “losing knowledge of a job”. The post is focused on PostgreSQL, but the core issue and takeaway are relevant to all data platforms, RDBMS especially.
I especially connected with this part:
I have the feeling that “we” are losing the knowledge of a job. I’ve come across experienced developers who didn’t even know that the DBA job existed. Recruiters asking me if, as a DBA, I knew how to optimize queries and if I knew SQL. That’s my job! It’s a bit like asking a plumber if he knows how to replace a tap!
What’s even more alarming is that I’m afraid there are also CTOs who lack this knowledge of data.
The consequence is that when performance problems arise, all we do is scale up the resources of the instances. With the cloud, the invoice grows linearly with the size of the instance. If load growth is exponential, the base will have to be scaled, and the bill with it. You end up with an exponential bill.
When you can’t cope with this, you’re going to blame the tool or the model, so you’re going to change DBMS or move to NoSQL. Spoiler: it won’t solve your problems.
Adrian Nayrat, Postgres again elected DBMS of the Year in 2023, but I’m worried
He’s not the only one writing about this. I’m starting to notice more and more people writing about this topic in their own words.
Indeed, we at Madeira Data Solutions are seeing this trend with our own eyes:
- More and more companies rely on both RDBMS and NoSQL platforms and look to hire multi-disciplinary employees who will take responsibility for all of these platforms.
- As a result, they lose the specialized expertise, and eventually suffer from performance and availability problems.
- Then, in search of this ‘lost expertise’, they turn to specialized consulting and outsourcing companies such as ours to fix their mistakes.
- As a result, demand for specialized consulting companies such as ours is on the rise.
- However, there are fewer and fewer DBAs willing to work for such specialized consulting companies, and it becomes harder and harder for specialized consulting companies to get new hires.
- Therefore, it becomes harder and harder for these specialized consulting companies to answer the ever-increasing demand for them.
This is a vicious cycle that’s only going to keep spiraling downward unless you and I do something to stop it.
Platform specialists and experts will be harder and harder to come by, and businesses are going to suffer more and more from bad performance, insufficient availability, limited scalability, and very, very, very, very dried-out wallets.
Where does it end? I honestly don’t know.
What Should We Do?
If you’re a C-level manager at a company such as those I described above, I urge you to take the following considerations when deciding what kind of person to hire:
- Part-time “In-Depth” Experts for Design and Planning:
- When building a new data system, hiring a part-time “in-depth” expert is crucial during the design and planning stage.
- I can’t stress this enough, but this stage is pivotal for laying down a solid foundation that is scalable, self-sustained, and optimized.
- It’s NOT necessary to have a full-time expert for such projects. Utilizing a part-time expert from a reputable consulting company is advisable, and cheaper in the long run.
- Part-time experts can provide specialized insights and solutions without the long-term commitment of a full-time hire.
- Transition to Full-time “Jack of All Trades”:
- Once the proper foundation for each data platform type is established, transitioning your hiring strategy toward a full-time “jack of all trades” makes more sense.
- Full-time hires can manage multiple platforms as part of their ongoing job responsibilities.
- Their focus shifts towards maintaining existing systems and incorporating incremental enhancements as the system matures and evolves.
- At this stage, in-depth knowledge becomes less of a necessity, and versatility in handling diverse platforms becomes more valuable.
This approach optimizes resource allocation by leveraging specialized expertise during critical stages and transitioning to an affordable and adaptable workforce for ongoing system maintenance and evolution.
By strategically timing the involvement of part-time specialists during the foundational stages and then transitioning to full-time jack-of-all-trades for ongoing maintenance and development, businesses can optimize their data management strategies while balancing the need for specialized expertise and versatility.
What Should You Do?
If you’re a DBA contemplating whether you should be a specialist in one platform, or a multi-disciplinary “jack of all trades” chasing a trend, consider the following:
Becoming a “Jack of All Trades”:
- For DBAs looking to diversify their careers, learning about different data systems can be a smart move.
- Being able to work with various systems makes you more valuable to a wider range of employers.
- It’s especially useful if you enjoy learning new things and riding the next trend.
- Take into consideration, though, that you’ll most likely not have the opportunity to truly master any one platform, and this means that you may be making many mistakes when you’re tasked with making significant changes.
Becoming an “In-Depth” Expert:
- For DBAs, specializing in one particular platform can also be a good move.
- It means you can become a go-to person for that specific system, which can open up some great job opportunities, especially in specialized consulting companies (or you could even become an independent freelancer).
- By focusing on one thing, you can become a real pro and stand out in a crowded field.
- You shouldn’t be afraid of becoming ‘irrelevant’ or having your job ‘phase out of fashion’. Especially if you’re specializing in an RDBMS platform. Those are here to stay for the long haul, despite all those perky NoSQL technologies coming and going.
- But you also need to take into consideration that fewer businesses will be willing to hire you as a full-time employee. On the other hand, you’ll be a great fit in specialized consulting and outsourcing companies.
Conclusion
Indeed, ladies and gentlemen, the current landscape of database administration presents a challenging dichotomy between the demand for multi-disciplinary skills and the necessity for in-depth expertise. While businesses increasingly seek DBAs who can work with a diverse selection of data platforms, there’s a notable risk in sacrificing specialization for versatility.
However, there is a balance to be struck by making use of both specialists and multi-disciplinaries at the right time and stage of development. And job seekers should feel safe choosing either path without fearing for their job security in the long term. Either approach will guarantee job security, although in different ways.
If this balance is where the market will be heading next, I think we would have a reason for being optimistic.