We are seeking an experienced Data Engineer to join our team. As a Data Engineer at GenPF, you will play a critical role in building and maintaining the infrastructure that powers our AI/ML and data science solutions. You’ll collaborate closely with our data scientists, machine learning engineers, and product team to design robust data pipelines and data architecture that support our financial analytics and predictive models.
Key Responsibilities
- Data Pipeline Development: Build, optimize, and maintain data pipelines for ingesting, processing, and storing large datasets from multiple financial sources, ensuring high availability and reliability.
- Data Architecture: Design and implement scalable, cloud-based data architecture, leveraging AWS services, to support real-time analytics and ML models.
- Data Integration: Work with external financial data providers, ensuring seamless integration and accuracy in data ingestion, transformation, and loading.
- Data Quality: Implement data validation and monitoring tools to ensure data integrity, accuracy, and consistency across all systems.
- Collaboration: Partner with data scientists and ML engineers to understand and implement data requirements for machine learning models and AI-driven financial insights.
- Automation & Optimization: Automate ETL processes and optimize data storage and retrieval for high performance, low latency, and cost-effectiveness.
- Documentation: Maintain detailed documentation of data infrastructure, data lineage, and workflows to ensure scalability and knowledge sharing within the team.
Required Qualifications
- Experience: 3+ years in data engineering, preferably within fintech, finance, or investment management industries.
- Proficiency in Python, SQL, and experience with data pipeline tools like Apache Spark, Apache Airflow, or similar.
- Expertise in AWS cloud services, including but not limited to S3, Redshift, Glue, and Lambda.
- Familiarity with data warehousing concepts and experience in designing scalable data architectures.
- Strong understanding of ETL processes, data wrangling, and data cleansing.
- Knowledge in Finance: Basic understanding of financial markets, portfolio management, and key financial datasets.
- Problem-Solving Skills: Ability to troubleshoot complex data issues and provide practical solutions.
- Collaboration: Excellent teamwork skills, with experience working cross-functionally with data science and engineering teams.
Preferred Qualifications
- Experience with ML/AI Pipelines: Familiarity with supporting ML operations and model data pipelines.
- DevOps Skills: Basic knowledge of CI/CD processes, containerization (Docker), and infrastructure as code (IaC) tools like Terraform or CloudFormation.
- Big Data Frameworks: Experience with frameworks like Hadoop or data lakes for large-scale data processing.
Benefits
- Competitive Salary & Equity: Opportunity to grow within a dynamic and rapidly growing AI/ML company.
- Remote/Hybrid Flexibility: Work from home or a location that suits you best.
- Professional Development: Access to learning resources, mentorship, and opportunities to work with leading AI and ML technologies.
- Work-Life Balance: Flexible work environment, recognizing the importance of balance and wellness.
Join GenPF.ai
Be part of an innovative company that’s shaping the future of portfolio management through advanced AI/ML and data science techniques. If you’re passionate about financial data, AI, and scalable data solutions, we’d love to hear from you!