Are you looking to work with an Agile, DevOps team culture? Are you looking to take your Cloud experience to their next challenge? If so, this role may be for you!
As a Cloud Data Engineer, you will work with the team to design and develop high-performance, resilient, automated data pipelines, streams, and applications, adapting technologies for ingesting, transforming, classifying, cleansing, and exposing data using creative design to meet objectives.
This is a contract-to-hire role (with full benefits).
Location: Greater Boston Area
- Build and maintain server less data pipelines in terabyte scale using AWS and GCP services.
- Integrate new data sources from enterprise sources and external vendors using a variety of ingestions patterns, including streams, SQL ingestion, file, and API.
- Work to develop and enhance the data architecture of the new environment, including recommending optimal schemas, storage layers, and database engines.
- Develop and implement data quality rules and logic across integrated data sources.
- Develop real-time data ingestion from a range of data integration sources, including business systems, external vendors and partners, and enterprise sources.
You Will Have:
- Master’s degree in computer science, mathematics, engineering, or equivalent work experience
- 4+ years with datasets with very high volume of records or objects
- Expert level programming experience in Python and SQL
- 2+ years working with Spark or other computer frameworks (Hadoop and Cloudera)
- 4+ years with relational databases (PostgreSQL, Microsoft SQL Server, MySQL, and Oracle)
- 2+ years with AWS Services including S3, Lambda, Redshift, and Athena
- Experience working with Google Cloud Platform Services
- Experience with contemporary data file formats like Apache Parquet and Avro
- Experience with streaming technologies (Amazon Kinesis and Kafka)
- Graph Database experience (Neo4j and Neptune)
- Distributed SQL query engines (Athena, Redshift Spectrum, and Presto)
- Experience with caching and search engines (Elasticsearch and Redis)
- ML experience (Amazon Sagemaker, DataRobot, and AutoML)
- IA coding tools, including CDK, Terraform CloudFormation, and Cloud Build
If you are interested, please apply today!
Smart Solutions, Inc. is an equal opportunity employer functioning under an Affirmative Action Plan.