Data Engineer

Company Description

Buluttan utilizes advanced Numerical Weather Prediction (NWP) and Artificial Intelligence models to create state-of-the-art hyper local weather forecasting solutions. Our products are designed to provide accurate and actionable weather intelligence. By building precision-driven solutions, we assist stakeholders in renewable energy and mobility industries,improving operational efficiency, and enhancing asset and people safety. Also consortium partner of the "EU Horizon: Weather generator" project, we are a vc-funded, curious team of engineers, driven to grow globally.

Role Description

We are seeking a talented and experienced Data Engineer to join our dynamic team. In this role, you will collaborate closely with our Data Science and Atmospheric Sciences Team to build and maintain robust data pipelines, ensure data quality, and support advanced analytics and machine learning initiatives. As a data company, our raw material is data and our final product is prediction data, which is why the Data Engineering team is primarily responsible for constructing pipelines that deliver data directly into our final products; hence, it is expected that you will also contribute to product design and engineering. You will contribute to the development and maintenance of our data pipelines, data lake, and data warehouse, as well as to building data products and tools that automate the processes.

Responsibilities:

• Design, develop, and maintain scalable data pipelines and ELT processes to handle large volumes of weather data
• Collaborate with Data Scientists and Meteorological Engineers to support data-driven projects and enhance predictive models
• Design and build data ingestion pipelines in Python to extract data from third-party sources, leveraging its libraries and tools to ensure efficient and accurate data retrieval
• Ensure data quality and integrity by implementing data validation and monitoring processes
• Optimize and manage our cloud-based data lake and warehouse architecture
• Administer our cloud services' permissions and security management, as well as building automation tools for permissions management and security processes
• Develop and maintain data models in our Python and SQL pipelines that directly deliver data to our end products, services, and ML and Atmospheric models
• Support product design and development by providing subject matter expertise regarding data pipelines, data models, and data architecture
• Assist in the creation of data visualizations and dashboards to provide insights for internal and external stakeholders
• Mentor and support junior engineers and scientists, fostering a collaborative and knowledge-sharing environment that enhances our data literacy and mindset

Requirements:

• Proven experience as a Data Engineer, preferably building and maintaining data pipelines that directly integrate with operational end products and services
• Strong proficiency in
  Python

and
SQL

and experience with relational databases and data warehouses (
BigQuery

, Redshift, MySQL, PostgreSQL) and data lakes (GCS, S3, Delta Lake)
- Hands-on experience with cloud-based solutions (
Google Cloud Platform

,

AWS, Azure), including data warehouses, data lakes, security and permissions management, and servers
- Understanding of modern data architecture concepts like data lakehouse, change data capture,  data marts, star/snowflake schemas, data mesh/fabric, and platform engineering
- Experience with data transformation tools (dbt, DataBricks), data orchestration tools (Airflow, AWS Glue, Bruin), and familiarity with monitoring and logging tools (DataDog, Logstash)
- Experience with Linux/Unix command line, including proficiency with shell scripting (bash, sh), process management, routine automation, and basic system administration tasks
- Proficiency with data modeling and transformation tools, with expertise in designing and implementing complex logics, develop quality checks, and optimize data schemas and partitioning for performance and scalability
- Knowledge of Object-Oriented Programming
- Collaboration skills in Github repositories using best practices, creating automation pipelines and actions, resolving conflicts, and managing deployment cycles
- Proven reputation of problem-solving techniques, attention to detail, stakeholder management, requirements gathering, and professional communication skills

Nice to Have:

• Knowledge of data visualization tools (Grafana, Tableau, Looker)
• Familiarity with APIs and extracting data from third-party sources
• Experience working with multi-dimensional data such as Zarr files and GeoJSON
• Experience with ingesting, processing, and converting image and radar files
• Passion for meteorology and working with weather data
• An active stance on climate change & sustainability

If you are passionate about data engineering and eager to make an impact with your skills, apply now and join our mission to leverage weather data for a better future.