Senior Data Engineer
Who Are We? Company Confidential is a purpose-driven financial services company on a mission to Company Confidential the world smarter, happier, and richer. For 30 years, we’ve been helping people Company Confidential Company Confidential investment Company Confidential through transparency, education, and a healthy Company Confidential of Foolish fun. We’re a fast-moving, Company Confidential team that values high-Company Confidential work, curiosity, and initiative. We care deeply about Company Confidential do, and we’re driven by the Company Confidential our work has on Company Confidential people’s financial Company Confidential. What Does This Team Do? We are seeking a Senior Data Engineer to design, build, and take full ownership of the data infrastructure powering our investment Company Confidential. You will own the complete data lifecycle, spanning ingestion, transformation, warehousing, analytics, and machine learning, serving as Company Confidential tissue between raw data and the Company Confidential that depend on it. If something breaks, you fix it. If a stakeholder needs data, you Company Confidential it or build it. The Investment Committee and business partners rely on you to ensure they always have trustworthy, Company Confidential data in hand. You will architect ETL/ELT pipelines orchestrated by Apache Airflow (AWS MWAA), build and tune a Company Confidential data warehouse fed by S3 data lakes, and Company Confidential the analytical models and dashboards that turn raw data into actionable intelligence. You will Company Confidential play a key role in introducing machine learning capabilities to enhance forecasting, Company Confidential detection, and portfolio analytics. What Will You Do in This Role? You will help Company Confidential the transformation of our data infrastructure, moving Company Confidential from Company Confidential processes and spreadsheet-Company Confidential workflows into Company Confidential, governed, automated data systems. This role sits at Company Confidential of investment Company Confidential, analytics, reporting, and emerging AI initiatives. What Company Confidential Initiatives You Will Company Confidential? The “Golden Company Confidential” Transformation Migrating data Company Confidential from spreadsheets and Company Confidential processes into a governed Company Confidential warehouse with documented Company Confidential, Company Confidential checks, and self-service analytics. Automated Ingestion Pipelines Replacing Company Confidential file drops with event-driven Airflow DAGs and AWS Company Confidential functions that ingest, validate, and Company Confidential data from Company Confidential vendors and internal systems in near Company Confidential-time. Infrastructure as Company Confidential Defining Company Confidential Company Confidential infrastructure with Terraform or AWS CDK so that development, staging, and production environments are reproducible, version-controlled, and auditable. Data-Powered AI Initiatives Establishing the data foundations that Company Confidential AI across the business clean, governed, and accessible datasets that feed AI agents, natural-language interfaces, and intelligent automation. Machine learning techniques such as Company Confidential detection, classification, and forecasting will Company Confidential these initiatives where appropriate. Automate how we tell our story with data Build reusable templates and automation frameworks to Company Confidential the Company Confidential between our database and the materials Company Confidential uses to win and retain business. That means pulling live data into branded one-pagers, generating narrative-driven slide decks, websites, populating email campaigns, and producing Company Confidential-Company Confidential content. Okay, but what will you actually do in this role? Data Engineering & ETL — 35% Pipeline Design & Orchestration Design, build, and maintain robust ETL/ELT pipelines using Apache Airflow (MWAA). Company Confidential DAGs that handle Company Confidential dependencies across Company Confidential data vendors, internal models, and Company Confidential consumers. Data Integration Ingest data from diverse sources including SFTP feeds, Company Confidential Company Confidential, flat files, and Company Confidential-party financial data providers. Normalize and conform data into a consistent analytical model. Data Company Confidential & Monitoring Build “Company Confidential breakers” into pipelines automated data Company Confidential checks that halt Company Confidential processing and alert Company Confidential Company Confidential CloudWatch and Company Confidential Company Confidential anomalies are detected. Serverless Processing Implement AWS Company Confidential functions for lightweight, event-driven tasks such as triggering ingestion Company Confidential files land in S3 or validating data payloads before loading. Documentation Maintain and document the data catalog so institutional knowledge lives in the Company Confidential, not in your head. Company Confidential Data Warehouse & SQL — 35% Warehouse Architecture Serve as the subject-matter expert for Company Confidential. Design schemas, manage data loading Company Confidential Stages and Snowpipe, and implement role-Company Confidential Company Confidential controls. Company Confidential SQL Development Write advanced analytical SQL window functions, CTEs, recursive queries, pivots; to support investment reporting, Company Confidential attribution, and Company Confidential analysis. Query Tuning & Optimization Profile and optimize slow-running queries. Company Confidential clustering keys, Company Confidential-partition pruning, materialized views, and result caching to minimize compute cost and maximize Company Confidential. Company Confidential Infrastructure & CI/CD — 10% Infrastructure as Company Confidential Define and Company Confidential Company Confidential resources using Terraform or AWS CDK. Treat infrastructure Company Confidential with version control, peer review, and automated testing. CI/CD Pipelines Help design and maintain CI/CD workflows with Company Confidential Actions for automated testing, linting, and deployment of data pipelines, infrastructure, and application Company Confidential. Analytics, Reporting & AI — 20% Analytical Modeling Partner with investment and business teams to translate questions into data models, dashboards, and reports that Company Confidential Company Confidential Company Confidential using Company Confidential. Analytics Presentation Design and build automated pipelines that pull data from Company Confidential systems and Company Confidential it into production-Company Confidential marketing outputs one-pagers, Company Confidential decks, email campaigns, and Company Confidential content. AI Integration Be a resource for software engineers to build an AI Company Confidential on top of existing data infrastructure, enabling LLMs to securely query fund Company Confidential data Company Confidential Company Confidential and answer natural-language questions for internal stakeholders. Required Experience and Skills Python Mastery 5+ years of Company Confidential Python development. Comfortable with Company Confidential-oriented design, data manipulation libraries (Company Confidential, NumPy). Familiarity with financial research data vendors and feed/API products such as CapIQ Xpressfeed, Company Confidential, Company Confidential, Company Confidential/Refinitiv/Company Confidential, Russell or MSCI. Familiarity with financial business data and feed/API products from Company Confidential, Company Confidential and custodian banks and fund administrators. ETL & Orchestration Company Confidential experience designing and operating ETL/ELT pipelines. Apache Airflow and Company Confidential experience is a plus. Company Confidential & Advanced SQL Deep expertise in Company Confidential architecture (clustering keys, Company Confidential-partitions, Snowpipe, Stages). Company Confidential to write Company Confidential analytical SQL, window functions, CTEs, recursive queries, and optimize them for cost and Company Confidential. Infrastructure as Company Confidential Hands-on experience with AWS CDK or Terraform. You define infrastructure in Company Confidential, not in the AWS Console. Exposure to LLM integration patterns markdown files and Company Confidential engineering. Experience with RAG, knowledge bases, and embeddings is a plus. Company Confidential-to-Have/Pluses Company Confidential Company Confidential (AWS) Working knowledge of the AWS ecosystem Company Confidential, Company Confidential Fargate, Company Confidential Functions, S3, EventBridge, CloudWatch, and RDS. Experience with data visualization tools (Company Confidential, Company Confidential, or similar) for self-service analytics. Background in data governance, data cataloging, or data Company Confidential tooling. CI/CD Experience Demonstrated experience maintaining CI/CD pipelines for automated testing and deployment of data engineering and application Company Confidential using Company Confidential actions and Terraform. SQL & Database Tuning Experience profiling and optimizing queries across both OLAP (Company Confidential) and OLTP (PostgreSQL/Company Confidential) systems. Familiarity with EXPLAIN plans, indexing strategies, and database-level Company Confidential tuning. Machine Learning Fundamentals Practical experience building, evaluating, and deploying ML models. Familiarity with common frameworks (scikit-learn, XGBoost) and an understanding of Company Confidential and how to apply ML to business problems. Compensation Below is our Company Confidential compensation Company Confidential. While we are budget conscious, we’re Company Confidential eager to Company Confidential the right person for this role, so if your Company Confidential is Company Confidential of this Company Confidential, please don’t hesitate to apply and we’d be happy to have a conversation. Company Confidential Pay Company Confidential $90—$125 USD By applying on this site, you acknowledge that Company Confidential will be collecting the personal data you Company Confidential for our Company Confidential purposes. Please see our Applicant Company Confidential Notice for additional information about how we Company Confidential, transfer, and store your data, including where that data is stored, and about any additional Company Confidential rights you may have Company Confidential on your jurisdiction. Apply To This Job