At Opensignal (Permanent), in Lisbon, PortugalExpires at: 2021-09-15Purpose of the roleWe’re looking for a Senior Data Engineer to join our Metrics Squad in transforming our real-world raw data into valuable and credible industry-leading metrics that provide insights to our analysts and our customers. You would be joining the team at a critical time as 5G begins to roll out across the globe and we lead the analysis into what this experience really looks like for real users.As a Senior Data Engineer, you will be responsible for:The creation and computation of our metrics. Working closely with our data scientists and our data engineers to create the metrics that are responsible for the pipeline that translates our raw data into usable insights that drive our business.As well as driving the creation of new 5G metrics, we are also looking at ways to optimise our pipeline and keep our tech stack fresh.We expect our Data Engineers to:Own and improve our data pipeline. Assemble large, complex data sets that meet business requirements, with engineering best practices in mind.Champion building scalable and resilient data infrastructure, as well as tools to extract and transform data used by stakeholders and customers.Be sensitive to GDPR concerns for data within the platform.Have a continuous improvement mindset when it comes to both the platform and the process.Work efficiently, automate manual processes where possible and take a test-driven approach to engineering.Take a keen interest in improving the platform’s scalability while understanding the cost.Be a good team player, with an agile approach and a can-do attitude.Keep yourself current and make sure we follow best practices and engineering standards.Be an advocate for the platform and its health. Take ownership of your work from conception through to support.Be detail orientated and understand the importance of the credibility of our metrics. Document and communicate with stakeholders in a language understood by all.Can work in a fast-paced environment with an ability to shift priorities and focus on changing requirements and market demandsAble to coach and mentor Data Engineers in best practicesCross-collaborate with the wider team to drive and maintain high standards in our data pipeline buildsComfortable and effective at working in a remote capacity, collaborating with team members across different locations through digital channelsMain requirementsRequirementsWhat we need from youAs a Data Engineer we would expect you to have previous experience in manipulating, processing, storing and extracting value from big data.Experience with big data tools, specifically Apache Spark.Experience with relational SQL databases. Prior experience with Postgres or desirably, AWS Athena (Presto).Experience with SQL query authoring.Experience with data pipeline / workflow tools i.e. Apache Airflow.Experience with AWS cloud services like EC2, S3, managed Kubernetes and Aurora.Experience with object-oriented/object function scripting languages: Python and desirably Scala.Experience in writing tests, especially in BDD style and working with Git.Strong analytic skills in working with unstructured datasets.Experience in root cause analysis of data when asked to answer specific business questions.Experience building and optimising 'big data' data pipelines.Experience supporting and working with cross-functional teams.Strong self-organisational skillsExperience being part of a cross functional team, using agile methodologies.Nice to haveExperience of having in a cross-functional team set up, ideally based on the Spotify Swaud model.Benefits & PerksCompetitive salary, stock options and on-target commission program to share our growth.Flexible working culture, constant exposure to learning opportunities in your role and the ability to make an impact from day one.£1000 (or local equivalent) learning budget with access to a learning platform providing access to thousands of opportunities to spend it on – from conferences to courses and training, and regular 'Opensessions' (internal learning talks).Regular all-company virtual events – ‘Openclubs’ (from our book club to photography club), regular company town halls and socials. We host quarterly meetings and bi-annual away days in-person with our global teams when we can get together again.The role will be home-based, and you’ll be supported to set up an effective home office environment.
