Tucker-Rose Associates is seeking to hire a Data Lake (Kafka) Engineer to help support a Federal Program within the US Department of Veterans Affairs.
SUMMARY:
Tucker-Rose Associates is seeking to hire a Data Lake (Kafka) Engineer to help support a Federal Program within the US Department of Veterans Affairs. We are partnered with Booz Allen on this position and will allow remote work.
RESPONSIBILITIES:
· Process and analyze both real-time streaming data and historical data to gain insights.
· Design data pipelines: Build robust, scalable, and secure data pipelines to ingest, process, and move data from various sources into the data lake using Kafka.
· Administer Kafka clusters: Deploy, configure, and maintain Kafka clusters and related ecosystem tools, such as Kafka Connect and Schema Registry, ensuring high availability and performance.
· Manage the data lake: Oversee the architecture and governance of the data lake, including managing data storage (e.g., in AWS S3 or ADLS), security, and metadata.
· Develop data processing applications: Create producers and consumers to interact with Kafka topics using programming languages like Python, Java, or Scala.
· Perform stream processing: Use tools like Kafka Streams, Apache Flink, or ksqlDB to perform real-time data transformations and analytics.
· Ensure data quality and security: Implement data quality checks, manage data lineage, and enforce security controls such as encryption, access controls (ACLs), and compliance.
· Monitor and troubleshoot: Set up monitoring and alerting for Kafka and data lake infrastructure and respond to incidents to ensure operational reliability.
REQUIREMENTS:
· Bachelor's Degree in Computer Science, Engineering, or Equivalent and 5 total years of experience; or 13 total years of experience in lieu of a degree
· 5+ years of experience with supporting, designing, and managing data platforms with Apache Kafka and big data technologies.
· Strong proficiency in languages like Python, Java, or Scala.
· Expertise in big data processing frameworks, such as Apache Spark and Apache Flink.
· Hands-on experience with cloud environments (AWS, Azure, or GCP) and relevant services like S3, Glue, or Azure Data Lake Storage.
· Data lake architecture: A solid understanding of data lake design principles, including storage formats (e.g., Delta Lake, Apache Iceberg), data modeling, and governance.
· Databases: Experience with various database systems, including both SQL and NoSQL.
· Infrastructure management: Familiarity with infrastructure-as-code tools like Terraform or Ansible and containerization with Docker and Kubernetes.
· Experience supporting federal agencies, preferable Veterans Affairs
CERTIFICATIONS
For Kafka:
· Confluent Certified Administrator for Apache Kafka (CCAAK) or Confluent Certified Developer for Apache Kafka (CCDAK)
For Data Lake and Cloud:
· Databricks Certified Data Engineer or AWS Certified Data Engineer or Microsoft Certified: Azure Data Engineer Associate