Math Coding and Analytics - abari212/data GitHub Wiki

Math Coding and Analytics

MCA is an innovative and inclusive platform that aims to address Big Data challenges spanning big data integration, Big Data preparation to Big Data analytics, combining mathematics with coding.

MCA Conceptual framework

MCA addresses both methodological and epistemic (data interpretation) challenges as well as the shortage of coding and Big Data skills. MCA follows a dual conceptual framework approach to address Big Data challenges, as research has revealed that Big data and Big data analytics challenges are not only methodological but also epistemological, for instance scaling up algorithms and inferences.

Application Development environment

MCA uses different platforms for coding to develop applications in the cloud and in the edges using R and Spark ecosystem. MCA coding procedure involves a combination of virtual with real practices. The coding process involves capturing and coding real, natural, and physical world. A meaningful, memorable and motivational platform for youth and data savvy professionals to apply coding and develop innovative applications, including ML applications!

R programming platform (data analytics)

R is a programming language and software development environment, with a wide variety of modelling tools for linear and nonlinear modelling, time-series analysis, classification and clustering. It is also highly extensible to process and analyse Big Data in combination with Hadoop and Spark.

Hadoop ecosystem for big data

Hadoop is a framework that uses a Distributed File System (HDFS) to split Big Data to manageable data sets and MapReduce to scale big data processing. Hadoop framework was conceived for distributed storage and distributed processing at Yahoo labs.  

Spark ecosystem

Spark works on the top of Hadoop, while Hadoop processes data on disk Spark processes data in memory. Spark and R are thus used together to scale and speed up Big Data analytics and data visualisation.

Reference

A book on Big Data Challenges appeared this year to address Big Data Challenges at https://www.amazon.com/dp/1521580928?ref_=pe_870760_150889320. A new book on coding for Big Data integration, preparation and analytics in under preparation.