Share
Up until recently, it was easy to define the responsibilities of a given IT role. After all, most IT experts were just plain software engineers. Of course, there were also other specialists – like database admins, IT admins, or security engineers. But at companies producing software, around 95% of the engineering staff were usually vanilla developers[1].
In 2021, the landscape is different. With the evolution of multiple technologies, the industry witnesses more complex and narrow specialization paths. Sure, the market is still filled with full stack developers or versatile experts able to combine different roles in an organization. But with the rising complexity of technology, IT professionals concentrate on one specific area more and more often. As a result, the last decade witnessed the development of multiple new expert roles.
In this article, I'm going to help you understand them.
[1] However, it's worth noting that they sometimes had also other responsibilities besides only developing software – like managing databases, creating automation tools, managing data, defining data analysis pipelines, creating integration components and more.
The Rise of New IT Roles
Many of these new roles are linked to the rise in popularity of certain technologies and solutions.
One of the most prominent examples is the Cloud. According to Statista, in 2020, many organizations that invested in the Cloud hired a wide array of experts skilled in narrow areas of expertise. Among these specialists were Cloud Architects, Cloud System Administrators, Security Architects, DevOps Engineers, Cloud Software Engineers, Cloud Systems Engineers, Cloud Network Engineers, Cloud Developers, Data Architects, Cloud Consultants, Security Engineers, Cloud Product Managers, Site Reliability Engineers, and... more!
Similarly, another widely varied and professionalized area is data engineering.
And these two are only examples.
With such a broad specialization variety, differentiating the ins and outs of many roles stopped being as easy as listing the responsibilities of an all-around software engineer. So, now, as promised, we'll take a look at some of the most popular new IT roles that you're bound to come across in most of the modern IT departments.[2] For this article, we've picked the roles of DevOps Engineer, Site Reliability Engineer, Cloud Developer, Data Engineer, and Data Scientist.
[2] However, please keep in mind that this is only our perspective. Since these roles are relatively new, you may also come across very different definitions.
DevOps Engineer
DevOps Engineers are involved primarily in automating software development lifecycles.
They mainly focus on:
- creating or supporting local environments for developers;
- managing deployment processes, preparing the deployment and release automation (in short, creating deployment pipelines);
- creating and managing branching strategies in code repositories;
- creating and managing infrastructure and its underlying configuration (usually stored in code);
- managing observability – logs management, infrastructure and application monitoring, etc.;
- managing overall system security.
DevOps engineers should be the ones delivering the business features. They are usually a full-fledged scrum team member with a specific set of responsibilities – and cooperate mostly with the Architect. Ideally, DevOps engineers should also have some basic programming skills, since most of the automation tasks involve coding.
A DevOps engineer’s skills are usually defined from 2 main perspectives. They include the following:
- Understanding the general concepts. This includes areas like SDLC, deployment types, networking, monitoring, testing, and more.
- Being proficient with tools. There are a lot of tools, some of which are critical in project work. Depending on the individual preferences, every DevOps engineer may have experience with a different set of tools. Luckily, these tools are usually more or less comparable. We can assume that if a DevOps Engineer knows Jenkins, they should also be able to work with GitlabCI (however, this doesn’t necessarily mean full proficiency).
The most popular buzzwords related to DevOps engineers include AWS, Azure, Azure DevOps, Terraform, ARM, CloudFormation, SAM, Jenkins, GitlabCI, Github, Bamboo, TeamCity, Kubernetes, AKS, EKS, GKE, ECS, Fargate, Ansible, Chef, Puppet, Docker, Vault, Helm, GitOps, Bash, PowerShell, and Python.
Site Reliability Engineer
A Site Reliability Engineering expert should have a similar skillset to a DevOps Engineer – however, with a slightly different focus. While a DevOps engineer is usually working on developing business features, an SRE expert is mainly responsible for maintenance and monitoring[3].
As a result, Site Reliability Engineers are, typically:
- monitoring infrastructure, debugging, measuring;
- monitoring applications;
- managing observability;
- reacting to incidents;
- migrate workloads, if needed.
The most popular buzzwords are similar to the DevOps role but with the addition of monitoring-related phrases like Zabbix, New Relic, DataDog, Nagios, Cloudwatch, ELK/EFK, Business Continuity, or Disaster Recovery.
[3] But remember that this differentiation isn’t set in stone. It may vary depending on the project and organization.
Cloud Developer
Cloud Developers have the ability to develop applications with managed cloud services.
They should be skilled in several areas, including:
- Delivering Business Features. A Cloud Developer should be proficient in a technology that enables to run applications in the cloud (with a special focus on the serverless approach). Sometimes, Cloud Developers need to work with multiple technologies even within the same project. As a result, Cloud Developers shouldn’t focus on one specific language or framework.
- Cloud proficiency. Cloud Developers should work comfortably with the cloud of their choice – both with basic services (like networking) and serverless. Usually, a Cloud Developer has one preferred cloud. They should possess relevant in-depth knowledge to eliminate mistakes and unoptimized code or infrastructure; this limits unwanted costs.
- DevOps. This area is usually needed only for deploying serverless applications. As a result, Cloud Developers should be proficient in, for example, AWS SAM. A Cloud Developer isn’t a DevOps Engineer in terms of managing deployment processes, running test environments, or similar. Instead, a Cloud Developer should be DevOps-aware – for example, to define the limitations and purpose of a pipeline.
Most popular buzzwords include Python, NodeJS, AWS, AWS Lambda, DynamoDB, S3, Fargate, Aurora, IAM, SSM, KMS, Kinesis, Glue, Athena, Azure, Azure Functions, CosmosDB, and many more managed service names.
Data Engineer
In short, a data engineer is often a DevOps for data.
As a result, their tasks usually include:
- orchestrating data ingestion in different contexts – data streaming, traditional databases, data lakes, and more;
- organizing and implementing data lakes;
- creating and managing data processing pipelines;
- managing data migration processes;
- creating and managing databases and data warehouses;
- creating reports and visualizations with business intelligence tools;
- diving deep into data (however, this task is often also performed by data analysts).
Regularly, data engineers perform their tasks in the cloud, so cloud proficiency is also required – especially including the tools that are specific for data engineering (like Athena, Glue, Redshift or QuickSight in AWS).
Moreover, data engineers should be familiar with DevOps tools to create and manage pipelines and cloud resources effectively.
The most popular data engineering buzzwords include AWS, Azure, GCP, Kubernetes, Athena, Glue, Kinesis, Redshift, RDS, Dataflow, Cloud Composer, Azure Data Factory, Apache Beam, Apache Airflow, Jupyter, QuickSight, Tableau, PowerBI, Looker, Python, Pandas, Terraform, Snowflake, Kafka, Flink, Cassandra, or MLOps.
Data Scientist
Data Scientists are skilled with Machine Learning (ML) algorithms. They don’t have to be familiar with cloud or DevOps technologies and don’t have to know how to set up a production-like environment. Instead, Data Scientists can analyze the data, observe patterns, and use the obtained knowledge to build and test ML models.
The created models can be of various types – like prediction or classification models. As a plus, a Data Scientist should be aware of how to deploy the models to enable inference (for instance, using the models as an API).
Most importantly, Data Scientists should excel at AI mathematics and techniques. They regularly work with Data Engineers who provide the data to work with and use the created models in the production phase – with help of MLOps techniques.
If you'd like to learn more about the work of data engineers and data scientists, read one of our previous articles: Data Engineer vs Data Scientist – What’s the Difference? | PGS Software
Summary
Our list of the most popular new IT roles should help you identify what kind of expert you need.
But don’t forget that there’s more – we only briefly described a few roles. Moreover, remember that many engineers like to cross the borders between defined responsibilities; they often look for new challenges.
Any questions? Don't hesitate to ask – I'll gladly answer your inquiries on LinkedIn or through the contact form!