I didn’t know what I wanted to do in my career for the longest time. I studied business in university, flip-flopping between concentrations, and doing multiple internships in various roles. And despite not knowing what I was going to do post-graduation, I still felt hopeful that I would figure it all out along the way.
At my first company, I decided to participate in a rotational program, one that would give new grads the opportunity to try different roles within the organization. I started out working in supply chain management, then revenue management, and finally marketing insights. At the surface, these roles don’t seem to have many common threads, but underlying them was data and lots of it.
I was absorbed by all the data that the company had, and at the same time, frustrated when I felt limited by what I could do. Sometimes I couldn’t make sense of how unruly the data was, how slow it was to run a simple calculation, or how undefined the data was. Despite the frustrations, I was equally if not more passionate to figure out how to make the process better. That’s when I discovered analytics engineering.
Anyone who has worked in a data role like data engineering or data analytics will have experienced some of the same frustrations I have. However, when you get to the root of these issues, these symptoms occur when there aren’t defined processes, communication, or resources to build a strong data foundation. Many companies build processes on top of poor data foundations, leading to insurmountable loads of tech debt, which only gets exasperated by an immature tech stack for data analytics. All that changed a few years ago with the introduction of dbt, springing the role of the analytics engineer.
An analytics engineer is meant to bridge the gap between data engineering and analytics. Per dbt, analytics engineers provide clean data sets to end users, modeling data to empower end users to answer their own analytics questions1.
You’re probably wondering, how does an analytics engineer differ from a data analyst and a data engineer? Any data analyst will tell you they spend way too much time fixing data on an ad-hoc basis, rather than bringing insights to their organization. And any data engineer will tell you they hate pulling data for end users in the requested form when they could be managing data flow processes. With the rise of the analytics engineer, an organization can have a designated individual that transforms, tests, deploys, and documents data. Analytics engineers apply software engineering best practices to the analytics code base1, giving way for modularity and quality assurance.
By this point, I was a few months into OSU’s computer science program. I loved everything that I was learning about software engineering, from version control to CI/CD, to agile development. And yet, I wasn’t completely drawn into working in software engineering because I still loved working with data.
I enjoy being able to improve processes to better enable others in their work. I love using the opportunity I have in analytics engineering to educate users on our data models. But above all else, I love how dynamic the role is. Sometimes there isn’t a clear distinction between a data engineer, a data analyst, and an analytics engineer, partly because of how new the role is. And that’s okay for me. Some days I might spend some time retooling a Looker dashboard. On other days I’m code-reviewing Python scripts that extract data from a source. But the best days are when I’m in dbt, automating a series of transformations to help us organize the data.
It took a few years before I found my calling in analytics engineering. And reflecting on the experience, I think it was a necessary one (partly because the role didn’t exist during my first degree). For anyone who is looking for an opportunity to be part of a modern data team, I would 100% recommend the role of analytics engineering. The tech stack is still evolving, and every day presents an opportunity to learn something new.
Sources:
1. https://www.getdbt.com/what-is-analytics-engineering/
Leave a Reply