Data Engineer with Python
4 weeks ago
- Develop and implement data pipelines for ingesting and collecting data from various sources into a centralized data platform.
- Develop and maintain ETL jobs using AWS Glue services to process and transform data at scale.
- Optimize and troubleshoot AWS Glue jobs for performance and reliability.
- Utilize Python and PySpark to efficiently handle large volumes of data during the ingestion process.
- Design and implement scalable data processing solutions using PySpark to transform raw data into a structured and usable format.
- Apply data cleansing, enrichment, and validation techniques to ensure data quality and accuracy.
- Create and maintain ETL processes using Python and PySpark to move and transform data between different systems.
- Optimize ETL workflows for performance and efficiency.
- Collaborate with data architects to design and implement data models that support business requirements.
- Ensure data structures are optimized for analytics and reporting.
- Work with distributed computing frameworks, such as Apache Spark, to process and analyze large-scale datasets.
- Manage and optimize databases, both SQL and NoSQL, to support data storage and retrieval needs.
- Implement indexing, partitioning, and other database optimization techniques.
- Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver effective solutions.
- Work closely with software engineers to integrate data solutions into larger applications.
- Implement monitoring solutions to track data pipeline performance and proactively identify and address issues
- Ensure compliance with data privacy regulations and company policies.
- Stay abreast of industry trends and advancements in data engineering, Python, and PySpark.
Requirements
- Proficiency in Python and PySpark.
- Strong knowledge of data engineering concepts and best practices.
- Hands-on experience with AWS Glue and other AWS services.
- Experience with big data technologies and distributed computing.
- Familiarity with database management systems (SQL and NoSQL).
- Understanding of ETL processes and data modeling.
- Excellent problem-solving and analytical skills.
- Strong communication and collaboration skills.
- Bachelor's degree in Computer Science, Information Technology, or a related field.
-
Python Developer for Probe
4 weeks ago
Beijing, China Software Full timeThe position Mobileye China R&D Center is looking for highly motivated AV Map Validation Engineer to make huge impact on the safety of advanced driving assistance systems (ADAS) and fully autonomous vehicles (AV).AV Map Validation Engineer at Mobileye have solid mathematical and algorithmic foundation in specific domain of graph, computer vision,...
-
Software Engineer
1 month ago
Beijing, China EDB Full timeA Little About UsInnovative, collaborative minds wanted. The world loves Postgres. We envision a world where organizations thrive by harnessing the full power of Postgres, the world’s fastest growing and most loved and used open source database. Our mission is to enable data teams everywhere to harness the full power of Postgres, whether on premises or in...
-
Perfect Engineer
4 weeks ago
Beijing, China Data Annotation Full timeThe position Mobileye's Perfect Engineer should understand Mobileye's map products and technologies, understand targeted technical issues raised by HQ and customers, collect, and organize relevant data, and continuously optimize the accuracy and precision of REM products. What will your job look like: Identify production map bug/issue and fix...
-
Intern-AI Research with Python-Beijing
1 day ago
Beijing, China Ericsson Full timeDescription Join our Team About this opportunity We are looking for a Digital Hardware Design Intern What you will do Be responsible for Digital Hardware design work in CCC Hardware Development Project. You will bring • Master or above degree and major in electrical engineering and related fields. • Enthusiastic about AI with a...
-
Intern-AI Research with Python-Beijing
14 hours ago
Beijing, China Ericsson Full timeDescription Join our Team About this opportunity We are looking for a Digital Hardware Design Intern What you will do Be responsible for Digital Hardware design work in CCC Hardware Development Project. You will bring • Master or above degree and major in electrical engineering and related fields. • Enthusiastic about AI with a...
-
Intern-AI Research with Python-Beijing
5 days ago
Beijing, China Ericsson Full timeDescription Join our Team About this opportunity We are looking for a Digital Hardware Design Intern What you will do Be responsible for Digital Hardware design work in CCC Hardware Development Project. You will bring • Master or above degree and major in electrical engineering and related fields. • Enthusiastic about AI with a...
-
Intern-AI Research with Python-Beijing
4 days ago
Beijing, China Ericsson Full timeDescription Join our Team About this opportunity We are looking for a Digital Hardware Design Intern What you will do Be responsible for Digital Hardware design work in CCC Hardware Development Project. You will bring • Master or above degree and major in electrical engineering and related fields. • Enthusiastic about AI with a...
-
Machine Learning Engineer
4 weeks ago
Beijing, China Zenni Optical Full timeZenni was founded on the idea that everyone who needs glasses should be able to afford high-quality frames and top-of-the-line lenses. By selling directly from our factories to our customers through our website we save them money and offer a huge selection of options. We have state-of-the-art production facilities and advanced prescription edging labs that...
-
Machine Learning Engineer
4 weeks ago
Beijing, China Zenni Optical Full timeZenni was founded on the idea that everyone who needs glasses should be able to afford high-quality frames and top-of-the-line lenses. By selling directly from our factories to our customers through our website we save them money and offer a huge selection of options. We have state-of-the-art production facilities and advanced prescription edging labs that...
-
Data Science Intern
4 weeks ago
Beijing, China WorldQuant Full timeWorldQuant develops and deploys systematic financial strategies across a variety of asset classes and global markets. We seek to produce high-quality predictive signals (alphas) through our proprietary research platform to employ financial strategies focused on market inefficiencies. Our teams work collaboratively to drive the production of alphas and...
-
Data Modeler
4 weeks ago
Beijing, China DBS Bank Full timeData Modeler-(WD56744) Responsibilities · Design data mart solutions in big data platform to store and retrieve company information · Analyze requirements for new requests on data mart enhancement · Migrate data analysis process from legacy systems to big data platform · Design conceptual and logical data models and flowcharts · Improve system...
-
Data Engineer
4 weeks ago
Beijing, China Unison Consulting Pte Ltd Full timeImplement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions Ensure data quality and integrity throughout the data processing lifecycle Design and...
-
Beijing, China Microsoft Full timeOverview We are inviting you to join the Mobile Experiences & Platform Team (MXP). The Mobile Experiences & Platform Team (MXP) is building the future. We are a world class platform, middleware, apps, and services organization that brings end-to-end experiences to life on a diverse range of form factors - empowering customers through powerful...
-
Data Enablement Specialist
4 weeks ago
Beijing, China DBS Bank Full timeData Enablement Specialist-(WD55058) Main Responsibility: Data enablement team’s mission is to prompt data usage in ADA lite platform. During this transformation journey, this position needs to take up product manager role to engage with BU/SU for 1) collecting their data requirements; 2) proposing executable solution; 3) collaborating with tech team and...
-
Data Scientist
4 weeks ago
Beijing, China Trexquant Investment Full timeAs a member of the Data team at Trexquant, you will be involved in parsing and analyzing large data sets, working on discovering and obtaining new sources of data, and collaborating with the Alpha and Strategy Research team to build predictive machine learning models.Responsibilities Explore and learn about a wide range of data sets that are used to develop...
-
Data Engineer
4 weeks ago
Beijing, China Unison Consulting Pte Ltd Full timeDesign, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in...
-
QA Engineer
7 days ago
Beijing, China Veeva Systems Full timeThe Role China SFA is a SaaS CRM product defined and built specifically in China, for China. We are looking for functional QA/Senior QA Engineers with BlackBox and automation testing experience to join the team. What You’ll Do Responsible for functional testing on the web, Enterprise WeChat, and iOS app Work closely with product managers and...
-
Computer Vision Lab Engineer
4 weeks ago
Beijing, China Algorithms Full timeThe position Mobileye is the global leader in the development of computer vision and machine learning, data analysis, localization and mapping for Advanced Driver Assistance Systems and autonomous driving. Mobileye’s technology helps keep passengers safer on the roads, reduces the risks of traffic accidents, saves lives and has the potential to...
-
Digitization engineer
4 weeks ago
Beijing, China HITACHI ENERGY HIGH VOLTAGE SWITCHGEAR CO., LTD., BEIJING Full timeDescription : Your responsibilities: You will identify the opportunities of digitalization by collecting and analyzing requirements from functions and transform them into digital project base on technology feasibility study and investment vs. benefit analysis. You will regularly review the operation status of digital system and continuous to...
-
Senior Data Scientist
4 weeks ago
Beijing, China Advance Intelligence Group Full time1. Working experiences in credit risk management;2. Good English skills, fluent in business communication;3. Good knowledge and understanding in risk management process, experiences in hands-on developing risk control model is a plus;4. Knowledge and clear expression in regular risk management modelling;5. Good communication skills and teamwork...