Lead Data Engineer Job at WorkHQ, Los Angeles, CA

azVXczJDM2FvTklxRFVMM0F0UFlIK2t3R3c9PQ==
  • WorkHQ
  • Los Angeles, CA

Job Description

Company Context

Series A, well-funded US startup in HRTech developing WorkHQ.com and an AI Recruiter product.

This is a US-only, Remote role (Mainland).

Role Overview

Lead data infrastructure architect managing billions of data points across 250M+ professional profiles.

Hire data engineers to aid you in that journey.

Core Responsibilities

  • Design scalable data pipelines processing massive record volumes

  • Architect ETL processes using PySpark on Amazon EMR (Open to shifting to other solutions like Data Bricks / Snowflake)

  • Distribute enriched data through medallion architecture across Postgres, Athena, OpenSearch

  • Integrate new data sources into the main pipeline

  • Implement advanced data matching using Splink

Technical Requirements

  • 5-8 years professional data engineering experience

  • Good proficiency in:

    • PySpark and distributed computing

    • AWS data services (EMR, Glue, Athena)

    • Docker

    • Pandas and DataFrame manipulation

    • Complex data format handling (JSONL, Parquet)

  • Strong background in:

    • Big data processing architectures

    • Data warehouse design

    • Performance optimization

  • Advanced Python, SQL skills

Nice to Have

  • Probabilistic record linking expertise

  • OpenSearch/elasticsearch technologies

  • Machine learning data pipeline design

  • Recruitment tech ecosystem knowledge

Technical Stack

  • Big Data: PySpark, EMR

  • Databases: Postgres, OpenSearch

  • Cloud: AWS

  • Containerization: Docker

  • Data Formats: JSONL, Parquet

  • Analytics: Metabase, Athena, Glue

  • Data Processing: Pandas, Splink

Other Considerations

While this role has specific requirements - if you lack a few technical skills, but motivated to learn and lead the platform, please apply for consideration.

If you are coming from Director/Head of/VP levels that is relevant to this job, you can apply as well.

You will need to apply directly on our platform.

Thank you for your time.

Job Tags

Permanent employment, Shift work,

Similar Jobs

Sher Careers

Sales Associate (Work From Home) Job at Sher Careers

 ...by reaching out to potential customers, building strong relationships, and guiding them through the sales process. This is a work-from-home opportunity that offers flexibility, independence, and a chance to grow within a supportive and business entrepreneur environment... 

Buffalo Wild Wings

Mixologist/Bartender Job at Buffalo Wild Wings

 ...providing the ultimate experience for our fans. And, of course, its our people who bring that experience to life GAME DAY As a bartender, we count on you to create a unique, personal experience for each Guest you encounter by making personal connections What brings... 

NBC Universal

Sous Chef Job at NBC Universal

 ...and have world-renowned theme parks and attractions through Universal Destinations & Experiences. NBCUniversal is a subsidiary of Comcast Corporation.Comcast NBCUniversal has announced its intent to create a new publicly traded company ('SpinCo') comprised of most of... 

EssilorLuxottica Group

Associate Doctor of Optometry-Ponce, Puerto Rico-LensCrafters Macys Job at EssilorLuxottica Group

 ...re committed to taking care of you, so you can bring the best quality experience to our patients and customers. LensCrafters and Macy's relationship is built around a shared mission of providing customers with the highest quality eye care, a passion for style and a... 

Senior Living Communities - Maxwell Group

Care Manager/MDS Coordinator/Registered Nurse Job at Senior Living Communities - Maxwell Group

 ...of something extraordinary! Now accepting applications for the MDS Coordinator - RN position. This position is responsible for optimal...  ...reason for a skilled service and reimbursement request. Skilled Nursing experience desired. We look forward to meeting you soon!...