Data Engineer Big Data

KochiApply for this jobJob DescriptionDesign, develop, and maintain scalable and efficient big data processing pipelines distributed computing systems.Collaborate with cross-functional teams to understand data requirements and design appropriate data solutions.Implement data ingestion, processing, and transformation processes to support various analytical and machine learning use cases.Optimize and tune data pipelines for performance, scalability, and reliability.Monitor and troubleshoot pipeline performance issues, identifying and resolving bottlenecks.Ensure data quality and integrity throughout the pipeline, implementing data validation and error handling mechanisms.Stay updated on emerging technologies and best practices in big data processing and analytics, incorporating them into our data engineering practices.Document design decisions, technical specifications, and data workflows.Tell us about youCandidate Name *Email *Mobile *+1India (भारत)+91United States+1United Kingdom+44Afghanistan (‫افغانستان‬‎)+93Albania (Shqipëri)+355Algeria (‫الجزائر‬‎)+213American Samoa+1Andorra+376Angola+244Anguilla+1Antigua and Barbuda+1Argentina+54Armenia (Հայաստան)+374Aruba+297Australia+61Austria (Österreich)+43Azerbaijan (Azərbaycan)+994Bahamas+1Bahrain (‫البحرين‬‎)+973Bangladesh (বাংলাদেশ)+880Barbados+1Belarus (Беларусь)+375Belgium (België)+32Belize+501Benin (Bénin)+229Bermuda+1Bhutan (འབྲུག)+975Bolivia+591Bosnia and Herzegovina (Босна и Херцеговина)+387Botswana+267Brazil (Brasil)+55British Indian Ocean Territory+246British Virgin Islands+1Brunei+673Bulgaria (България)+359Burkina Faso+226Burundi (Uburundi)+257Cambodia (កម្ពុជា)+855Cameroon (Cameroun)+237Canada+1Cape Verde (Kabu Verdi)+238Caribbean Netherlands+599Cayman Islands+1Central African Republic (République centrafricaine)+236Chad (Tchad)+235Chile+56China (中国)+86Christmas Island+61Cocos (Keeling) Islands+61Colombia+57Comoros (‫جزر القمر‬‎)+269Congo (DRC) (Jamhuri ya Kidemokrasia ya Kongo)+243Congo (Republic) (Congo-Brazzaville)+242Cook Islands+682Costa Rica+506Côte d’Ivoire+225Croatia (Hrvatska)+385Cuba+53Curaçao+599Cyprus (Κύπρος)+357Czech Republic (Česká republika)+420Denmark (Danmark)+45Djibouti+253Dominica+1Dominican Republic (República Dominicana)+1Ecuador+593Egypt (‫مصر‬‎)+20El Salvador+503Equatorial Guinea (Guinea Ecuatorial)+240Eritrea+291Estonia (Eesti)+372Ethiopia+251Falkland Islands (Islas Malvinas)+500Faroe Islands (Føroyar)+298Fiji+679Finland (Suomi)+358France+33French Guiana (Guyane française)+594French Polynesia (Polynésie française)+689Gabon+241Gambia+220Georgia (საქართველო)+995Germany (Deutschland)+49Ghana (Gaana)+233Gibraltar+350Greece (Ελλάδα)+30Greenland (Kalaallit Nunaat)+299Grenada+1Guadeloupe+590Guam+1Guatemala+502Guernsey+44Guinea (Guinée)+224Guinea-Bissau (Guiné Bissau)+245Guyana+592Haiti+509Honduras+504Hong Kong (香港)+852Hungary (Magyarország)+36Iceland (Ísland)+354India (भारत)+91Indonesia+62Iran (‫ایران‬‎)+98Iraq (‫العراق‬‎)+964Ireland+353Isle of Man+44Israel (‫ישראל‬‎)+972Italy (Italia)+39Jamaica+1Japan (日本)+81Jersey+44Jordan (‫الأردن‬‎)+962Kazakhstan (Казахстан)+7Kenya+254Kiribati+686Kosovo+383Kuwait (‫الكويت‬‎)+965Kyrgyzstan (Кыргызстан)+996Laos (ລາວ)+856Latvia (Latvija)+371Lebanon (‫لبنان‬‎)+961Lesotho+266Liberia+231Libya (‫ليبيا‬‎)+218Liechtenstein+423Lithuania (Lietuva)+370Luxembourg+352Macau (澳門)+853Macedonia (FYROM) (Македонија)+389Madagascar (Madagasikara)+261Malawi+265Malaysia+60Maldives+960Mali+223Malta+356Marshall Islands+692Martinique+596Mauritania (‫موريتانيا‬‎)+222Mauritius (Moris)+230Mayotte+262Mexico (México)+52Micronesia+691Moldova (Republica Moldova)+373Monaco+377Mongolia (Монгол)+976Montenegro (Crna Gora)+382Montserrat+1Morocco (‫المغرب‬‎)+212Mozambique (Moçambique)+258Myanmar (Burma) (မြန်မာ)+95Namibia (Namibië)+264Nauru+674Nepal (नेपाल)+977Netherlands (Nederland)+31New Caledonia (Nouvelle-Calédonie)+687New Zealand+64Nicaragua+505Niger (Nijar)+227Nigeria+234Niue+683Norfolk Island+672North Korea (조선 민주주의 인민 공화국)+850Northern Mariana Islands+1Norway (Norge)+47Oman (‫عُمان‬‎)+968Pakistan (‫پاکستان‬‎)+92Palau+680Palestine (‫فلسطين‬‎)+970Panama (Panamá)+507Papua New Guinea+675Paraguay+595Peru (Perú)+51Philippines+63Poland (Polska)+48Portugal+351Puerto Rico+1Qatar (‫قطر‬‎)+974Réunion (La Réunion)+262Romania (România)+40Russia (Россия)+7Rwanda+250Saint Barthélemy+590Saint Helena+290Saint Kitts and Nevis+1Saint Lucia+1Saint Martin (Saint-Martin (partie française))+590Saint Pierre and Miquelon (Saint-Pierre-et-Miquelon)+508Saint Vincent and the Grenadines+1Samoa+685San Marino+378São Tomé and Príncipe (São Tomé e Príncipe)+239Saudi Arabia (‫المملكة العربية السعودية‬‎)+966Senegal (Sénégal)+221Serbia (Србија)+381Seychelles+248Sierra Leone+232Singapore+65Sint Maarten+1Slovakia (Slovensko)+421Slovenia (Slovenija)+386Solomon Islands+677Somalia (Soomaaliya)+252South Africa+27South Korea (대한민국)+82South Sudan (‫جنوب السودان‬‎)+211Spain (España)+34Sri Lanka (ශ්‍රී ලංකාව)+94Sudan (‫السودان‬‎)+249Suriname+597Svalbard and Jan Mayen+47Swaziland+268Sweden (Sverige)+46Switzerland (Schweiz)+41Syria (‫سوريا‬‎)+963Taiwan (台灣)+886Tajikistan+992Tanzania+255Thailand (ไทย)+66Timor-Leste+670Togo+228Tokelau+690Tonga+676Trinidad and Tobago+1Tunisia (‫تونس‬‎)+216Turkey (Türkiye)+90Turkmenistan+993Turks and Caicos Islands+1Tuvalu+688U.S. Virgin Islands+1Uganda+256Ukraine (Україна)+380United Arab Emirates (‫الإمارات العربية المتحدة‬‎)+971United Kingdom+44United States+1Uruguay+598Uzbekistan (Oʻzbekiston)+998Vanuatu+678Vatican City (Città del Vaticano)+39Venezuela+58Vietnam (Việt Nam)+84Wallis and Futuna (Wallis-et-Futuna)+681Western Sahara (‫الصحراء الغربية‬‎)+212Yemen (‫اليمن‬‎)+967Zambia+260Zimbabwe+263Åland Islands+358Experience *Experience In Years 0 years 1 year 2 years 3 years 4 years 5 years 6 years 7 years 8 years 9 years 10 years 11 years 12 years 13 years 14 years 15 years 16 years 17 years 18 years 19 years 20 years 21 years 22 years 23 years 24 years 25 years 26 years 27 years 28 years 29 years 30 years Experience In YearsExperience In Months 0 months 1 month 2 months 3 months 4 months 5 months 6 months 7 months 8 months 9 months 10 months 11 months Experience In MonthsPreferred Locations *Select Country India USA Canada UAE China Japan Vietnam Netherlands United Kingdom Select CountrySelect States Select StatesSelect Cities Select CitiesCurrent Location *Select Country AfghanistanAland IslandsAlbaniaAlgeriaAmerican SamoaAndorraAngolaAnguillaAntarcticaAntigua And BarbudaArgentinaArmeniaArubaAustraliaAustriaAzerbaijanBahrainBangladeshBarbadosBelarusBelgiumBelizeBeninBermudaBhutanBoliviaBonaire, Sint Eustatius and SabaBosnia and HerzegovinaBotswanaBouvet IslandBrazilBritish Indian Ocean TerritoryBruneiBulgariaBurkina FasoBurundiCambodiaCameroonCanadaCape VerdeCayman IslandsCentral African RepublicChadChileChinaChristmas IslandCocos (Keeling) IslandsColombiaComorosCongoCook IslandsCosta RicaCote D'Ivoire (Ivory Coast)CroatiaCubaCuraçaoCyprusCzech RepublicDemocratic Republic of the CongoDenmarkDjiboutiDominicaDominican RepublicEast TimorEcuadorEgyptEl SalvadorEquatorial GuineaEritreaEstoniaEthiopiaFalkland IslandsFaroe IslandsFiji IslandsFinlandFranceFrench GuianaFrench PolynesiaFrench Southern TerritoriesGabonGambia TheGeorgiaGermanyGhanaGibraltarGreeceGreenlandGrenadaGuadeloupeGuamGuatemalaGuernsey and AlderneyGuineaGuinea-BissauGuyanaHaitiHeard Island and McDonald IslandsHondurasHong Kong S.A.R.HungaryIcelandIndiaIndonesiaIranIraqIrelandIsraelItalyJamaicaJapanJerseyJordanKazakhstanKenyaKiribatiKosovoKuwaitKyrgyzstanLaosLatviaLebanonLesothoLiberiaLibyaLiechtensteinLithuaniaLuxembourgMacau S.A.R.MadagascarMalawiMalaysiaMaldivesMaliMaltaMan (Isle of)Marshall IslandsMartiniqueMauritaniaMauritiusMayotteMexicoMicronesiaMoldovaMonacoMongoliaMontenegroMontserratMoroccoMozambiqueMyanmarNamibiaNauruNepalNetherlandsNew CaledoniaNew ZealandNicaraguaNigerNigeriaNiueNorfolk IslandNorth KoreaNorth MacedoniaNorthern Mariana IslandsNorwayOmanPakistanPalauPalestinian Territory OccupiedPanamaPapua new GuineaParaguayPeruPhilippinesPitcairn IslandPolandPortugalPuerto RicoQatarReunionRomaniaRussiaRwandaSaint HelenaSaint Kitts And NevisSaint LuciaSaint Pierre and MiquelonSaint Vincent And The GrenadinesSaint-BarthelemySaint-Martin (French part)SamoaSan MarinoSao Tome and PrincipeSaudi ArabiaSenegalSerbiaSeychellesSierra LeoneSingaporeSint Maarten (Dutch part)SlovakiaSloveniaSolomon IslandsSomaliaSouth AfricaSouth GeorgiaSouth KoreaSouth SudanSpainSri LankaSudanSurinameSvalbard And Jan Mayen IslandsSwazilandSwedenSwitzerlandSyriaTaiwanTajikistanTanzaniaThailandThe BahamasTogoTokelauTongaTrinidad And TobagoTunisiaTurkeyTurkmenistanTurks And Caicos IslandsTuvaluUgandaUkraineUnited Arab EmiratesUnited KingdomUnited StatesUnited States Minor Outlying IslandsUruguayUzbekistanVanuatuVatican City State (Holy See)VenezuelaVietnamVirgin Islands (British)Virgin Islands (US)Wallis And Futuna IslandsWestern SaharaYemenZambiaZimbabweSelect CountrySelect State Select StateSelect City Select CityLinkedIn Profile URLResume File * (PDF files only)How did you hear about this job opening? * Select... Litmus7 Website Social Media Referred by a Litmus7 employee Riverside Chat Podcast Billboard Advert Others Select...Primary Skills * Others .Net AEM AI AI/ ML Akamai Analytics - Snowflake Android Android Angular JS Apache Kafka Apache SOLR Apache Storm API Testing Apigee Appium Application Support ATG ATG ATG Support ATG-BCC ATG-CSC AWS Azure Backbone JS Big Data Big Data - Hadoop Platform Bootstrap JS Business Analyst Calabash Cassandra Charles Client Engineering Cloud - AWS Cloud - Azure Cloud - GCP Cloud Data Analytics solutions (Looker, QuickSight) Cloud engineering Commerce1 - Commerce Tools Commerce1 - HCL Commerce Commerce1 - Oracle Commerce Commerce1 - Retisio Commerce Commerce2 - Adobe Commerce2 - BigCommerce Commerce2 - Magento Commerce2 - SAP Hybris Commerce2 - SFCC Commerce2 - Shopify Commerce2 - Support Communication Create React App Creative Graphic Designer CSS Data Engineer Data engineer Data Warehousing Database Concepts Dayforce Default Delivery Manager DemandWare DevOps DevOps DevOps Digital Marketing Endeca Espresso ETL Fiddler Finance and Accounts Fluent Commerce Flutter Full Stack Full-Stack Hadoop HCM (Human Capital Management) Hibernate HR HTML Hybris Informatica infosec iOS iOS Jasmine JS Java Java Core Java MS Java MS Search Java/J2EE JavaScript Jira jQuery Kotlin Lambda Logical reasoning Magento Magento Mainframe as400 Maven Micorsoft Dynamics AX Microservices Microservices1 - Java Microservices2 - .Net Microservices2 - ESB Microservices2 - NodeJS Microservices2 - Python Microsoft Dynamics 365 Business Central Microsoft Dynamics NAV Mobility - AR/VR Mobility - Flutter Mobility - iOS, Android Mobility - React Native Mocha JS MySQL NaVision Netsuite Network Network Support Networking Networking Node JS NoSQL Objective-C OMS OMS - Blue Yonder OMS - Sterling OMS - Yantrix OMS service Operation Manager Oracle Database Passion for Technology Performance Test Engineer Performance Tester PHP POS POS PoS - JumpMind Pos - Oracle Xstore Power BI Problem Solving / Logical Thinking Product Development Management Product Management Program manager Programming Basics Programming Skills Project Management Project Manager Python QA QA Automation QA Automation QA Manual QA1 - Manual Testing QA1 - Test Automation QA2 - Perf Engineering QA2 - Security Testing Qlikview React Golang React JS React Native React Native Recruitment Recruitment Redux JS RESTful Web Services Retail Domain Robotic Process Automation RPA RPA Ruby Salesforce CRM SAP Saucelabs Scala Scrum Scrum Master Scrum Master SDET Selenium SEO SFCC SFCC / Demandware SFMC- Salesforce Marketing Cloud SFSC- Salesforce Service Cloud Shell Scripting Shopify solr Spring Spring boot SQL SQL Server SRE SSIS SSRS Sterling Sterling Call Center Struts Supply Chain - JDA Supply Chain - Manhattan Supply Chain - NetSuite Support Swift System Administer Tableau Technical Writer TestNG Tomcat UI UI - Angular UI - HTML, CSS, JS UI - React UI - UX UI Node UI REACT Vespa Visual Studio Team Services (VSTS/TFS) VMware Vue.JS vue.js WCS WCS WebMethods WebPack WebServices WFM (Workforce Management) WMS wordpress Wordpress Xamarin XCode XCUITest Others .Net AEM AI AI/ ML Akamai Analytics - Snowflake Android Android Angular JS Apache Kafka Apache SOLR Apache Storm API Testing Apigee Appium Application Support ATG ATG ATG Support ATG-BCC ATG-CSC AWS Azure Backbone JS Big Data Big Data - Hadoop Platform Bootstrap JS Business Analyst Calabash Cassandra Charles Client Engineering Cloud - AWS Cloud - Azure Cloud - GCP Cloud Data Analytics solutions (Looker, QuickSight) Cloud engineering Commerce1 - Commerce Tools Commerce1 - HCL Commerce Commerce1 - Oracle Commerce Commerce1 - Retisio Commerce Commerce2 - Adobe Commerce2 - BigCommerce Commerce2 - Magento Commerce2 - SAP Hybris Commerce2 - SFCC Commerce2 - Shopify Commerce2 - Support Communication Create React App Creative Graphic Designer CSS Data Engineer Data engineer Data Warehousing Database Concepts Dayforce Default Delivery Manager DemandWare DevOps DevOps DevOps Digital Marketing Endeca Espresso ETL Fiddler Finance and Accounts Fluent Commerce Flutter Full Stack Full-Stack Hadoop HCM (Human Capital Management) Hibernate HR HTML Hybris Informatica infosec iOS iOS Jasmine JS Java Java Core Java MS Java MS Search Java/J2EE JavaScript Jira jQuery Kotlin Lambda Logical reasoning Magento Magento Mainframe as400 Maven Micorsoft Dynamics AX Microservices Microservices1 - Java Microservices2 - .Net Microservices2 - ESB Microservices2 - NodeJS Microservices2 - Python Microsoft Dynamics 365 Business Central Microsoft Dynamics NAV Mobility - AR/VR Mobility - Flutter Mobility - iOS, Android Mobility - React Native Mocha JS MySQL NaVision Netsuite Network Network Support Networking Networking Node JS NoSQL Objective-C OMS OMS - Blue Yonder OMS - Sterling OMS - Yantrix OMS service Operation Manager Oracle Database Passion for Technology Performance Test Engineer Performance Tester PHP POS POS PoS - JumpMind Pos - Oracle Xstore Power BI Problem Solving / Logical Thinking Product Development Management Product Management Program manager Programming Basics Programming Skills Project Management Project Manager Python QA QA Automation QA Automation QA Manual QA1 - Manual Testing QA1 - Test Automation QA2 - Perf Engineering QA2 - Security Testing Qlikview React Golang React JS React Native React Native Recruitment Recruitment Redux JS RESTful Web Services Retail Domain Robotic Process Automation RPA RPA Ruby Salesforce CRM SAP Saucelabs Scala Scrum Scrum Master Scrum Master SDET Selenium SEO SFCC SFCC / Demandware SFMC- Salesforce Marketing Cloud SFSC- Salesforce Service Cloud Shell Scripting Shopify solr Spring Spring boot SQL SQL Server SRE SSIS SSRS Sterling Sterling Call Center Struts Supply Chain - JDA Supply Chain - Manhattan Supply Chain - NetSuite Support Swift System Administer Tableau Technical Writer TestNG Tomcat UI UI - Angular UI - HTML, CSS, JS UI - React UI - UX UI Node UI REACT Vespa Visual Studio Team Services (VSTS/TFS) VMware Vue.JS vue.js WCS WCS WebMethods WebPack WebServices WFM (Workforce Management) WMS wordpress Wordpress Xamarin XCode XCUITestI confirm that the data shared in this application are accurate and truthful to the best of my knowledgePlease check the declaration to proceed with submissionBack Show more Show less