Data Engineer - DevOps

About The Company :

CrowdANALYTIX is a crowdsourced analytics service focused on partnering with life sciences and professional services firms globally. CrowdANALYTIX operates a platform in which a large community of independent analytical experts solve business problems by competing in data science competitions. CrowdANALYTIX currently has data scientists on its platform from 50 countries, many with PhDs and Masters in Statistics and Machine Learning. CrowdANALYTIX is backed by Accel Partners and SAIF Partners and is based out of Silicon Valley, California.

Website : https://www.crowdanalytix.com/

Job Location : Bangalore

Educational Qualification : B.Tech/M.Tech (CS/IT/EC/EE), BCA/MCA, B.Sc/M.Sc. (IT)

Desired Experience : 0 - 6 months

Salary : INR 1.80 LPA to INR 3.6 LPA

Probation Period : 6 months

Bond : No 

Job Description & Skill Set Required :

- Development of large-scale real-time web data crawling system and storage platform The data could be from reviews, blogs, product catalogs, social sites, travel data- basically anything and everything that's publicly available. 

- For the crawling of HTML or XML files, be able to use wrapper applications. 

- Convert the extracted unstructured web data, convert them in to the structured data in NoSQl 

- Managing multiple servers (several EC2 / other servers). 

- Would include maintaining the system health, monitoring, upgrading and patching softwares, writing scripts to automate day to day tasks and scaling the infrastructure as per the requirements. 

- Systems are mainly Linux/Unix based and the other tools / databases could be varied.

Interview Process :

- 2 Technical Rounds

- 2 HR Rounds