Tools & Technologies Used in AI ML - tech9tel/ai GitHub Wiki
๐ ๏ธ Tools and Technologies in AI/ML
๐งฐ Programming Languages
Language | Purpose | Notes / Usage |
---|---|---|
Python | General-purpose, AI/ML/DL | Most widely used for ML, NLP, CV, automation. Popular for AI, ML, and DL due to extensive libraries (e.g., TensorFlow, PyTorch). Commonly used for model building, data manipulation, and scientific computing. |
R | Statistical computing | Popular in academic and research ML settings. Widely used in statistics, data analysis, and machine learning. Suitable for research and academic projects. |
Java | Enterprise-grade applications | Used in large-scale backend ML services. Commonly used in large-scale enterprise applications, backend services, and scalable ML systems. |
C++ | High-performance computing | Ideal for robotics, real-time AI systems. Used in performance-critical applications, including some AI model implementations and game development. |
JavaScript | Web-based AI | Used with TensorFlow.js and browser ML |
Julia | Scientific computing | High performance for numerical and large datasets. Known for high-performance numerical computing, often used for complex mathematical models and algorithms. |
Scala | Big Data + ML integration | Used with Apache Spark MLlib. Used for big data analytics and distributed computing, especially in combination with Spark. |
Swift | iOS & Apple AI | CoreML, mobile AI development |
Go (Golang) | Backend and scalable ML systems | Efficient concurrency and cloud-based ML apps |
๐งช Frameworks and Libraries
Name | Category | Language | Use Case / Description |
---|---|---|---|
TensorFlow | DL Framework | Python, C++ | Training deep learning models, production-ready. Open-source framework by Google for deep learning and AI development. Used for creating neural networks, training models, and deploying in production. |
PyTorch | DL Framework | Python | Preferred in academia, dynamic graphs. Widely used deep learning framework by Facebook, known for its dynamic computational graph and flexibility. |
Scikit-learn | ML Library | Python | Classical ML algorithms and tools. Popular library for traditional machine learning algorithms such as regression, classification, clustering, etc. |
Keras | DL API | Python | Simplified interface over TensorFlow. High-level neural networks API built on top of TensorFlow, making it easier to create deep learning models. |
XGBoost | Gradient Boosting | Python, R | Structured/tabular data performance |
LightGBM | Gradient Boosting | Python, R | Fast and memory-efficient gradient boosting |
Hugging Face | NLP/Transformers | Python | Pretrained LLMs (BERT, GPT, etc.) |
FastAI | High-level DL API | Python | Simplifies PyTorch, quick experimentation |
OpenCV | Computer Vision | C++, Python | Image/video processing, real-time CV |
Dlib | CV & ML Toolkit | C++, Python | Face recognition, object tracking |
NLTK | NLP Toolkit | Python | Language parsing, tokenization, tagging |
SpaCy | NLP Toolkit | Python | Fast and production-grade NLP |
Gensim | Topic Modeling | Python | Word2Vec, document similarity |
Transformers | LLM/NLP Models | Python | Hugging Face transformer models |
PaddlePaddle | DL Framework | Python | Industrial-grade DL by Baidu |
Theano | DL Backend (Legacy) | Python | Historical backend, now mostly deprecated |
Apache MXNet | DL Framework | Python etc | A flexible and efficient deep learning framework known for supporting both symbolic and imperative programming. |
๐งฑ Tools and Platforms
Tool / Platform | Type | Description / Usage |
---|---|---|
Jupyter Notebook | IDE / Notebook | Interactive ML prototyping, visualization |
Google Colab | Cloud Notebook | Free cloud GPU/TPU for ML experiments |
Kaggle | Platform & Notebook | ML datasets, competitions, and kernels |
Amazon SageMaker | ML Platform | Full ML pipeline deployment and management |
Azure ML Studio | ML Platform | Drag-and-drop ML development and automation |
Databricks | ML + Big Data | Unified platform for Spark + ML workflows |
MLflow | ML Lifecycle | Experiment tracking, model registry |
Weights & Biases | Experiment Tracker | Visualization, tracking, and collaboration |
DVC | Data Versioning | Git-like versioning for datasets and models |
Ray | Distributed ML | Scalable ML training and hyperparameter tuning |
Streamlit | ML App UI Builder | Turns Python scripts into shareable web apps |
Gradio | ML Demo Interface | Simple UI to demo models with text, image, audio |
๐ Data Processing Tools
Tool | Usage |
---|---|
๐ Pandas | Library for data manipulation and analysis, particularly used for handling structured data (e.g., CSV, Excel files). |
๐ข NumPy | Core library for numerical computing in Python, providing support for arrays, matrices, and mathematical operations. |
๐ Dask | Scalable analytics framework for parallel computing in Python, used to handle big data tasks that pandas can't manage efficiently. |
๐ฅ Spark | A unified analytics engine for big data processing, widely used for real-time data streaming and large-scale ML jobs. |
โ๏ธ Cloud Platforms for AI/ML
Platform | Usage |
---|---|
๐ Google Cloud AI | Offers a suite of machine learning services, including pre-trained models, APIs, and tools for developing custom AI models. |
๐ ๏ธ AWS AI Services | Provides a comprehensive suite of AI and machine learning services, including SageMaker for model development and deployment. |
๐ Microsoft Azure AI | Offers machine learning, deep learning, and AI services with tools for model training, deployment, and management. |
๐ง IBM Watson | Provides cloud-based AI tools for NLP, visual recognition, and machine learning. |
๐ AI/ML Tools for Visualization
Tool | Usage |
---|---|
๐ Matplotlib | Python library for creating static, animated, and interactive visualizations in Python. |
๐ผ๏ธ Seaborn | Built on top of Matplotlib, it provides a high-level interface for drawing attractive statistical graphics. |
๐ Tableau | Data visualization tool commonly used for business intelligence and generating interactive dashboards. |
๐ Power BI | Microsoftโs analytics and data visualization tool, which is widely used in the business world. |
๐ฃ๏ธ AI/ML Libraries for Natural Language Processing (NLP)
Library | Usage |
---|---|
๐ NLTK | A comprehensive library for processing human language data (text), offering tools for tokenization, parsing, and stemming. |
๐งโ๐ป spaCy | Advanced NLP library designed for production use. Supports tasks like named entity recognition, part-of-speech tagging, and dependency parsing. |
๐ฌ Transformers (Hugging Face) | A library for state-of-the-art NLP models like BERT, GPT-3, etc., widely used for tasks such as text classification, translation, and summarization. |
๐ Emerging AI/ML Technologies
Technology | Usage |
---|---|
๐ฎ Quantum Computing | Expected to revolutionize computing, potentially enabling faster and more powerful models. |
๐ก Federated Learning | A decentralized machine learning technique where data remains on devices, and only model updates are shared, enhancing privacy. |
๐ง AutoML | A set of tools and frameworks that automate the process of applying machine learning to real-world problems, making it easier for non-experts. |
๐ Neural Architecture Search (NAS) | A technique to automatically discover the optimal neural network architecture for a given task. |