Data Engineer job description
A Data Engineer builds the pipelines and storage that move data from apps and systems into a form the business can trust. A company needs one so reports, dashboards and models rest on accurate, timely data.
Why the role exists
The Data Engineer makes data usable. In Indian SaaS, fintech and e-commerce companies, data sits in app databases, payment systems, CRMs and spreadsheets. This person collects it, cleans it, stores it in a warehouse and makes sure analysts and data scientists can query it without delays or doubt.
What the person does
- Build pipelines that move data from app databases, APIs and files into a central warehouse.
- Clean and transform raw data into well-defined tables that analysts can use without rework.
- Schedule and monitor jobs with Airflow or similar tools, and fix failures quickly.
- Add checks for missing, late or duplicate data and alert the owners when something looks wrong.
- Design warehouse tables and keep a simple data dictionary that explains each field.
- Tune slow queries and pipelines to keep processing time and cloud cost under control.
- Work with analysts, data scientists and product teams to understand what data they need.
- Protect sensitive customer data through access controls and masking as the employer's policy requires.
Skills and tools
- SQL at an advanced level, including joins, window functions and tuning
- Python for data processing and pipeline code
- Workflow tools such as Airflow, dbt or similar
- Warehouses such as BigQuery, Redshift, Snowflake or PostgreSQL
- Big data tools such as Spark or Kafka where the volume needs them
- Cloud storage and services on AWS, Azure or GCP
- Data modelling, including facts, dimensions and slowly changing records
- Explaining data issues in plain words to non-technical colleagues
Education and experience
Preferred, never a bar: equivalent experience counts.
- Preferred: a degree in computer science, statistics, engineering or a related field, or equivalent experience.
- Preferred: experience building and running at least one data pipeline in production.
- Candidates from backend, BI or database administration roles are welcome with proof of pipeline work.
- Preferred: awareness of data privacy practices, with details confirmed against the employer's own policy and adviser.
How the work is judged
- Reliability of pipelines, including how often data arrives complete and on time.
- Accuracy of the data, judged by reconciliation checks and complaints from analysts.
- Speed and cost of processing as data volumes grow.
- Clarity of documentation and how easily others can use the tables built.
- Responsiveness to requests from analytics, product and business teams.
Where the role can lead
- Senior Data Engineer, owning data platform design and standards.
- Data Architect or Analytics Engineering Lead, shaping how the company models its data.
- Machine Learning Engineer or Engineering Manager for a data team.
Interview questions to ask
Ask every candidate the same questions in the same order and score them against written anchors, so the decision rests on the work and not the impression.
- Describe a pipeline you built. Where did the data come from and what went wrong most often?
- A dashboard shows yesterday's sales as zero. How do you trace the problem?
- How do you design a table so that history is kept when a customer changes their address?
- Explain how you would load a large file daily without creating duplicate rows.
- A query takes an hour to run. What steps do you take to speed it up?
- How do you decide between processing data in batches and in near real time?
Hiring tips for this role
- Test SQL with a real problem, such as removing duplicates or finding the latest record per customer. Weak SQL is the commonest hidden gap.
- Ask what happened when a pipeline failed in production. Candidates who have run pipelines can describe retries, alerts and backfills.
- Do not confuse a Data Engineer with a Data Analyst or Data Scientist. Be clear on whether you need pipelines, reports or models.
- Check how they treat sensitive data. Careless handling of customer information can harm the business and its customers.
Questions about hiring a data engineer
A Data Engineer builds and maintains the systems that collect and store data. A Data Analyst uses that data to answer business questions and make reports. Small firms sometimes expect one person to do both.
When reports depend on manual exports, data sits in many tools, or analysts spend most of their time cleaning data. Before then, a skilled analyst or backend developer may cover the need.
Only if your data volume or speed needs them. Many companies run well on SQL, a cloud warehouse and a scheduler. Ask for the tools you will actually use, not the longest list.
Yes. GullyHR supports permanent hiring. We agree the job description with you, find and screen candidates against it, and support the offer stage. You decide who to hire.
Hiring a data engineer?
GullyHR runs talent acquisition for permanent roles. We agree the job description with you, find and screen candidates against it, and support the offer through to joining. We recruit permanent employees. We do not supply contract or temporary staff.