Data Engineer

Data Engineer
Employment term: Full-time
Annual Base Salary (ZAR): 0
Job responsibilities:

Zapier is a startup, so you’ll likely get experience on many different projects across the organization. Here are some things you’ll get a taste of:

  • Develop ETL to ingest and transform data from upstream databases and APIs into a data warehouse. The tools used include AWS Redshift, NiFi, Kafka, Matillion ETL, and custom Python.
  • Build, deploy, and continuously improve the infrastructure used by our data scientists and data and business analysts. The tools we have been using here include Docker, Terraform, Ansible, Kubernetes, and AWS EC2.
  • As a part of Zapier’s all-hands philosophy, help customers via support to ensure they have the best experience possible.

ABOUT DATA AT ZAPIER

Zapier relies on dozens of systems that emit data about Zapier and our potential and current users and partners. This data is useful for us to make a better product, better decisions, and understand our weaknesses and opportunities. The data team at Zapier pulls this data from DBs, APIs, and event streams, collocates it and then processes it through all the disparate systems to bring them together in a reliable, timely, performant, and easy to understand way for the employees and systems that need it.

Within the data team, we’re made up of several subteams: Data Ops focusing on data infrastructure and storage, compute, ingest, and dimensional modeling; Data Products focusing on building statistical and ML tools and models; and Data Governance.

Our stack is best summed up by: AWS Redshift (and the related AWS products AWS Glue, Redshift Spectrum, AWS S3), Looker, Airflow, Matillion ETL, Kafka, Python, and NiFi. But we’re pragmatic — for example, we have some Java for ingesting data from Kafka, and we use Clojure for inferring schema and other information about data sets.

Job requirements:
  • You have experience in SQL: You’ve used written complex SQL queries that join across data from multiple systems, matching them up even when there was not a straightforward way to join the tables. You’ve designed tables with an eye towards ease of use and high performance. You’ve documented schemas and created data dictionaries.
  • You are a skilled written communicator. Zapier is a 100% remote team and writing is our primary means of communication.
  • You appreciate our team’s values of eagerness to collaborate with teammates from any function of the organization or with any level of data knowledge, iterating over your deliverables, and being curious.
  • You understand that the perfect is the enemy of the good and default to action by shipping MVP code and iterating as needed to get towards better solutions.

WE’RE LOOKING FOR SOMEBODY WHO WOULD MAKE A SUCCESSFUL DATA ENGINEER…

  • You have experience with APIs: You’ve ingested large quantities of data from RESTful APIs.
  • You have experience and a comfort level with programming. You can read and write code in Python, Go, Rust, Java, or C#. You’re familiar with distributed source control using Git.
  • You have experience running infrastructure needed to orchestrate data pipelines, store data with different retention and performance requirements, and perform compute for multiple loads. Experience with tools like Ansible, Terraform, and/or Vagrant is a plus.
  • You understand columnar-store file formats like Orc or Parquet and are also familiar with Avro and Avro schemas.
Tags: SQL, RESTful API, Java, Git

Hi there!

We’re looking for a Data Engineer to join the data team at Zapier. Interested in helping grow a product that helps the world automate their work so they can get back to living? Then read on…

We know applying for and taking on a new job at any company requires a leap of faith. We want you to feel comfortable and excited to apply at Zapier. To help share a bit more about life at Zapier, here are a few resources in addition to the job description that can give you an inside look at what life is like at Zapier. We hope you’ll take the leap and apply.

Zapier is proud to be an equal opportunity workplace dedicated to pursuing and hiring a diverse workforce.

Sign In


Register


Reset Password

Please enter your username or email address, you will receive a link to create a new password via email.