Research On Domain Analysis - bounswe/bounswe2024group10 GitHub Wiki
SPARQL (pronounced "sparkle") stands for SPARQL Protocol and RDF Query Language. It is a powerful query language and protocol used to retrieve and manipulate data stored in RDF (Resource Description Framework) format. RDF is a standard model for data interchange on the web, providing a way to describe resources and their relationships in a machine-readable format.
SPARQL provides a query language that allows users to express queries against RDF datasets. These queries can range from simple searches to complex operations involving multiple datasets and conditions.
SPARQL also includes a protocol for communication between clients and servers, enabling applications to interact with remote RDF stores. This protocol defines how queries are sent to the server and how results are returned to the client.
The basic unit of data in RDF is a triple, consisting of a subject, predicate, and object. SPARQL queries are constructed using triple patterns, which specify conditions that must be met by matching triples in the dataset.
SPARQL queries can include variables, which are placeholders for values that will be matched against the data in the dataset. When a query is executed, these variables are bound to specific values found in the dataset, resulting in a set of bindings that satisfy the query conditions.
SPARQL queries can return results in various formats, including XML, JSON, and CSV. This flexibility allows developers to integrate SPARQL queries into a wide range of applications and systems.
SELECT Clause: Specifies the variables to be returned in the query results. WHERE Clause: Defines the conditions that must be met by matching triples in the dataset. Modifiers: Optional clauses that allow users to control the behavior of the query, such as limiting the number of results or ordering them based on certain criteria.
SELECT ?subject ?predicate ?object
WHERE {
?subject ?predicate ?object
}
LIMIT 10
This query selects the subject, predicate, and object of all triples in the dataset and limits the results to the first 10 matches.
SPARQL is widely used in the development of semantic web applications, which aim to make web content more accessible and understandable to both humans and machines.
SPARQL can be used to query and integrate data from multiple sources, allowing users to perform complex analyses and gain insights from diverse datasets.
Many knowledge graphs, such as DBpedia and Wikidata, provide SPARQL endpoints that allow users to query the data contained within these graphs and retrieve relevant information.
SPARQL plays a crucial role in the Linked Data initiative, which seeks to interconnect datasets on the web and enable seamless navigation between related resources.
- Web application development is defined as creating software and web-based applications that run on a web server and can be accessed over the internet using web browsers. This process involves designing, developing, testing, and deploying web-based systems that allow users to interact with data and perform tasks online.
-
A web application is an interactive program that runs on a web server and is accessed through a web browser. A web app is built so that the user interface provides data back to the development team that designed it. This data offers insights into customer interests, usage, and preferences that can prove invaluable to product and marketing strategies. The data can also inform optimization and other client-centered aspects of the mobile app or desktop applications.
-
A big difference between web applications and webpages is that user experience dictates web application design. Traditional web design is based on server-side programmers making decisions about what might enhance usability. In contrast, a web application has an application program interface (API) that takes in large-scale data from the user side and then funnels that information into automation.
-
For example, if a company has implemented an e-commerce web application that can track and report on which products a potential customer lingered over the longest, similar items can be suggested by the web application the next time the customer logs in. If that same company only had a website, the user’s experience would be based on what was available to every customer: a static and standardized set of options.
-
Most mobile applications that appear on smartphones are web apps. Here’s a brief list of common web applications:
- Facebook, Instagram, Twitter, and other social media platforms
- Gmail, Yahoo, AOL, and any web-based email programs
- Any self-service customer portal
- Query sites, such as Quora and Google
-
It’s important to note the distinctions that exist between native web apps and mobile apps. Mobile apps live on the device and are designed to run on a specified platform (such as iOS or Android). Examples would include Facebook Messenger and Google Maps. A Google search that takes you directly to the internet browser would be an example of a web app, while Google Maps is a mobile app.
-
Remember that web applications are all about input, and traditional sites are all about output. If you want customers to enter their information, you’re likely looking at a web application. If you just want customers to read static information that they don’t contribute to as a user, a webpage might better suit your business needs.
-
Developing a web app is all about setting goals for the app’s purpose. What need does the app fulfill? The user interface should be designed with that answer in mind. Information about the consumer will come from the user interface, so developers should design the app to receive and respond to that information. Web development entails tasks like:
- Making sure the web app offers compatibility with both Android and iOS
- Identifying life cycle and optimization metrics
- Building an intelligent, iterative user interface
- Progressive web applications are a newer type of web application which behaves like and often outperforms native applications. They are web applications which follow a slightly different methodology, and involve an additional set of technologies such as service workers, manifests, push notifications. Progressive web applications can be downloaded to your device, and stored on your homescreen which makes them accessible, and ’native-like’, and unlike web applications they can be accessed and used offline.
- You can look this youtube link for further information about python web applicatin development
- For general knowledge you can look this youtube tutorial
Mobile app(lication) development is the process used to create software for smartphones, tablets, and mobile devices. The overall process includes writing code to create the software and designing the app.
The major points of mobile application is to design the software compatible with mobile phones' infrastructure such as battery, hardware, operating system (which is usually Android or iOS), etc.
-
Native mobile apps: They are applications developed to target either Android or iOS. Depending on which operating system you are developing for, your applications are usually coded in a specific programming language.
-
Cross-Platform mobile apps: With this approach, one create cross-platform mobile applications from a single codebase. The goal of cross-platform app development is to target different operating systems with one project.
-
Pros of Native Development
-
- Better performance: You create and optimize native apps for a specific platform. You also compile them using the platform’s core programming language and APIs. This makes them very fast, more efficient, and responsive to users’ actions.
-
- Tight security: Native apps can enhance the security of your user’s data. They have access to platform-specific built-in security features.
-
- Quality UX: Native apps have a smoother user input and output experience. The look and feel are consistent because they inherit the device’s OS interfaces. They follow design guides for the specific OS, making the flow of the app feel more natural. Full feature set access: Native mobile apps can access all features and functionalities of the device, providing a richer, more integrated user experience. They also gain access to services like push notifications, which are key to driving user engagement.
-
- Minimal bugs: Native app developers get access to updated SDKs as soon as they are released. These updates always ship with improvements and new features.
-
Cons of Native Development
-
- Cost: Native mobile applications are usually costly to deploy. You need to have separate teams of developers for the specific platforms. For example, you need developers proficient in Java or Kotlin to create a native Android app. You will also need extra Swift/Objective-C developers for native iOS.
-
- Development time: Separate teams working on similar apps for separate platforms take time. Maintenance and enhancements also require a lot of effort.
-
- Lack of code reusability: You have to create and keep code in separate projects to target different mobile operating systems.
-
Pros of Cross-Platform Development
-
- Low costs: Cross-platform mobile development needs only a single team of developers. They need to be knowledgeable in the chosen framework. The app reaches a wider audience to start gaining traction and testing the market.
-
- Code reusability: Cross-platform frameworks allow you to use a single codebase. This ensures consistency in the business logic used.
-
- Rapid development: By reusing code and improving productivity, you get to market faster. Cross-platform frameworks have also optimized the process of app testing during development.
-
- Easier maintenance: Updating a cross-platform app is easier. This is because you are dealing with a single codebase. All builds targeting the different platforms use the same code, ensuring consistency.
-
Cons of Cross-Platform Development
-
- Larger digital footprint: Cross-platform apps are usually larger. You need to pay attention to the mobile optimization options available.
-
- Difficult integrations: Cross-platform frameworks do not integrate all platform-specific features. Some hardware-specific integrations like using GPUs might need skills in native app development.
-
- Lower performance: Cross-platform frameworks usually bundle a custom runtime to run your app. It is responsible for communicating with platform-specific services. This adds another layer of computation, which can lead to lower performance.
-
- Delayed platform features: New SDK releases typically include new features and updates. With cross-platform frameworks, you have to wait for a separate update to access these newer features.
- User interface (UI) design is the process designers use to build interfaces in software or computerized devices, focusing on looks or style. Designers aim to create interfaces which users find easy to use and pleasurable. UI design refers to graphical user interfaces and other forms—e.g., voice-controlled interfaces.
They usually come in three formats:
- Graphical user interfaces (GUIs)—Users interact with visual representations on digital control panels. A computer’s desktop is a GUI.
- Voice-controlled interfaces (VUIs)—Users interact with these through their voices. Most smart assistants—e.g., Siri on iPhone and Alexa on Amazon devices—are VUIs.
- Gesture-based interfaces—Users engage with 3D design spaces through bodily motions: e.g., in virtual reality (VR) games.
There are some basic conventions for a better UI design:
- Users judge designs quickly and care about usability and likeability.
- UIs should also be enjoyable (or at least satisfying and frustration-free).
- UIs should communicate brand values and reinforce users’ trust.
- User experience (UX) design is the process design teams use to create products that provide meaningful and relevant experiences to users.
- It involves the design of the entire process of acquiring and integrating the product, including aspects of branding, design, usability and function.
Here are some UX design conventions:
- UX designers consider the "Who", "Why", "What", and "How" of product use
- It is user-centered
- UI design is more concerned with the surface and overall feel of a design. UI design is a craft where you the designer build an essential part of the user experience. UX design covers the entire spectrum of the user experience.
- One analogy is to picture UX design as a car with UI design as the driving console.
- A mobile app backend is the brain of a mobile app. Among other things, the backend takes care of data processing, storage, and security.
- The backend operates on the server, and it’s that part of the app that you don’t see, but your mobile app depends on it for functionality.
A mobile backend takes care of:
-
Data processing and storage independent of a smartphone’s capabilities
-
Data sync and sharing across multiple devices and platforms
-
Content updates within the mobile app
-
Management of the app’s business logic
-
Authorization and authentication that control access to data
-
An API — application program interface — is used in mobile apps just like it is in web apps. It allows developers to access another application or platform.
Here's how APIs fit into mobile application development:
Here are some benefits of mobile APIs:
- Consumer Apps: For some apps — especially consumer ones — mobile APIs provide the tool that enables user interaction with data from multiple applications.
- Manufacturing Apps: In a traditional context (desktop, on-premises), an API will help you bring together functionality from a lot of different apps.
- Enterprise Apps: Many enterprise apps are more singularly focused. But they still require that authentication and security be locked down to protect corporate intellectual property. In all instances, the API (or multiple APIs) strengthen the application fundamentals: by allowing it to be flexible (whereby developers can use, and add to, its functionality), secure, yet all the while, totally usable.
- One of the primary responsibilities of a mobile app developer is handling data within his/her applications. Efficient data management is crucial for ensuring a seamless user experience, maintaining app performance, and safeguarding user privacy.
Some local data storage solutions are explained below:
- SQLite: SQLite is a lightweight and widely used relational database management system that can be integrated into mobile apps. It offers excellent performance, supports complex queries, and is available for both iOS and Android platforms.
- SharedPreferences/UserDefaults: For storing small amounts of key-value data, SharedPreferences (Android) or UserDefaults (iOS) can be used. These are lightweight and ideal for simple settings and preferences.
- File Storage: If your app deals with large files or unstructured data, such as images, audio, or video, consider using file storage mechanisms. Utilize appropriate directory structures and optimize file access for efficient data retrieval.
Here are some possible security measurements for safe data storage:
- Use HTTPS and SSL/TLS: Always transmit data over secure connections using HTTPS with SSL/TLS encryption. This ensures data integrity and prevents eavesdropping.
- Employ Authentication and Authorization: Implement robust authentication mechanisms to prevent unauthorized access. Utilize token-based authentication, OAuth, or other industry-standard protocols to authenticate users and control access to sensitive data.
- Encrypt Stored Data: If your app stores sensitive user information locally, employ encryption techniques to safeguard the data. Utilize algorithms like AES or RSA for encryption and securely manage encryption keys.
Here are some good practices for efficient data synchronization:
- Use Incremental Updates: Instead of transferring complete datasets, implement incremental updates where only modified or new data is synchronized. This reduces bandwidth usage and improves synchronization speed.
- Handle Conflict Resolution: Handle conflicts that may arise during data synchronization, such as concurrent edits or deletions. Implement conflict resolution strategies to ensure data consistency across devices.
- Implement Background Sync: To provide a seamless user experience, perform data synchronization in the background. Use background services or push notifications to trigger synchronization when new data is available.
- Regularly Backup and Test Data Restoration: Data loss can be catastrophic for users. Implement regular data backups on both the client and server sides. Additionally, periodically test data restoration processes to ensure data integrity and recovery in case of any unforeseen events or disasters.
- All popular mobile platforms provide security controls designed to help software developers build secure applications. However, it is often left to the developer to choose from myriad of security options. A lack of vetting can lead to security feature implementation that can be easily circumvented by attackers.
Common issues that affect mobile apps include:
- Storing or unintentionally leaking sensitive data in ways that it could be read by other applications on the user’s phone.
- Implementing poor authentication and authorization checks that could be bypassed by malicious applications or users.
- Using data encryption methods that are known to be vulnerable or can be easily broken.
- Transmitting sensitive data without encryption over the Internet.
Here are some possible security measures:
- Encrypting the source code
- Using a code-signing certificate
- Leveraging penetration testing
- Securing data in transit
- Only using authorized APIs
- Securing the backend
- Being careful with third-party libraries
- Deploying tamper detection
- Practising continual testing and updating
- A brief explanation about mobile app development can be found here.
- Native vs Cross-Platform Development
- UI/UX design
- Backend
- Data Storage and Management
- Security
Linked Data is structured data interlinked with other data, making it more useful through queries on the internet. It builds upon standard Web technologies such as HTTP, RDF, and URIs. Instead of merely displaying web pages and data for human readers, linked data extends them to be visible and share information in a way that can be read automatically by computers.
One of the visions of the linked data concept is for the Internet to become a global database, making interconnected data widely accessible. Note that no single entity owns or controls all Linked Data. It's a collaborative effort of organizations, individuals, and institutions. By using RDF, websites can provide structured and semantically rich data, enabling better interoperability between different systems and applications.
Linked data is represented by a graph data structure, providing more flexibility at the entity level compared to tabular data representation. Google has Knowledge Graph, and Facebook has OpenGraph for this purpose.
RDF, which stands for Resource Description Framework, is a framework for representing information on the web. It is a specification developed by the World Wide Web Consortium (W3C) for describing resources on the web in a machine-readable way. Note that not every website on the internet might adopt RDF standards.
Linked data concept is built upon RDF triples. An RDF triple follows the Subject - Predicate - Object pattern:
- A. Painter painted A Nice Day
- A Painter has the gender female
For relations and results to be meaningful, each node should be unique so that two different clients will not see different results related to that node.
Every authority has a unique universal resource identifier (URI). Checking Virtual International Authority File (VIAF), we can see that all authorities have a unique ID.
Triple Store is a purpose-built database to store RDF triples.
SPARQL is the language to query and retrieve RDF triples.
Below is a simplified example of an HTML5 code snippet for a biography page that adopts RDF principles using JSON-LD (Json Linked Data)
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Biography Page</title>
<script type="application/ld+json">
{
"@context": "https://schema.org",
"@type": "Person",
"name": "John Doe",
"birthDate": "1990-01-15",
"birthPlace": {
"@type": "Place",
"name": "Cityville, USA"
},
"alumniOf": {
"@type": "CollegeOrUniversity",
"name": "University of Example"
},
"jobTitle": "Web Developer",
"worksFor": {
"@type": "Organization",
"name": "Example Company"
},
"description": "John Doe is a web developer with a passion for creating interactive and user-friendly websites.",
"image": "john-doe.jpg"
}
</script>
</head>
<body>
<header>
<h1>John Doe</h1>
</header>
<section>
<h2>About Me</h2>
<p>John Doe is a web developer with a passion for creating interactive and user-friendly websites.</p>
</section>
<section>
<h2>Biographical Information</h2>
<ul>
<li><strong>Birth Date:</strong> January 15, 1990</li>
<li><strong>Birth Place:</strong> Cityville, USA</li>
<li><strong>Alma Mater:</strong> University of Example</li>
</ul>
</section>
<section>
<h2>Professional Details</h2>
<ul>
<li><strong>Job Title:</strong> Web Developer</li>
<li><strong>Works For:</strong> Example Company</li>
</ul>
</section>
<section>
<h2>Image</h2>
<img src="john-doe.jpg" alt="John Doe">
</section>
</body>
</html>
API stands for Application Programming Interface. It is a set of rules and protocols that allow one software application to interact with another.
APIs play a crucial role in enabling isolated systems and software to work together.
-
RESTful APIs:
REST stands for Representational State Transfer. It uses standard HTTP methods such as GET, POST, DELETE, etc. Currently, RESTful APIs are the most popular type for web and mobile applications due to their simplicity and scalability.
-
GraphQL APIs:
GraphQL provides clients with the option to retrieve only the data they need, offering more flexibility than REST APIs. It is also more efficient and reduces over-fetching.
-
Authentication APIs:
Authentication APIs simplify the authentication and authorization process for applications.
-
Payment APIs:
APIs that provide a set of functions and methods for online payments. Developers can integrate online payment systems into their mobile or web applications. Examples include Stripe and PayPal APIs.
To utilize a REST Api, we should send request to server which serves the API we want to use. Let us dive deeper to api request:
A REST (Representational State Transfer) API request is typically composed of several componenst. Here are the key components of a REST API request:
-
Endpoint (URL):
- Definition: The URL that uniquely identifies the resource or service the client wants to interact with.
-
Example:
https://api.example.com/users
-
Example:
https://api.example.com/profile
-
Example:
https://api.example.com/items/493827
-
Example:
https://api.example.com/items?name=fridge&minPrice=2000
-
HTTP Method (Verb):
- Definition: Type of operation the client wants to perform on the resource. Common HTTP methods include GET, POST, PUT, PATCH, and DELETE.
-
Example:
GET
to retrieve data,POST
to create a data on the resource,PUT
orPATCH
to update an existing resource, andDELETE
to remove a resource.
-
Headers:
- Definition: Additional information sent with the request, typically in the form of key-value pairs. Headers can include authentication tokens, content types, and more.
-
Example:
Authorization: Bearer your_access_token (With this token, API can detect that client is authenticated so it performs the operations client asked.) Content-Type: application/json
-
Parameters (Query Parameters or Path Parameters):
- Definition: Additional data sent with the request to modify the request or filter the response. Parameters can be included in the URL (query parameters) or within the URL path (path parameters). API can internally resolve this path parameters and performs the operations based on this query parameters.
-
Example (Query Parameters):
https://api.example.com/users?role=admin (**Meaning: Get the users which have admin role)
-
Example (Path Parameters):
https://api.example.com/users/{id} (**Meaning: Get the user with the given id)
-
Request Body:
-
Definition: Contains data that the client wants to send to the server, typically used in
POST
,PUT
, andPATCH
requests. The format of the data (e.g., JSON or XML) is specified in theContent-Type
header. -
Example:
POST
Request that creates a new user:{ "username": "john_doe", "email": "[email protected]", "password": "secretpassword" }
-
Definition: Contains data that the client wants to send to the server, typically used in
-
Authentication:
- Definition: Information used to authenticate the client's identity. Common methods include API keys, OAuth tokens, or basic authentication.
-
Example: Using an API key:
Authorization: ApiKey your_api_key
These components define the main structure and purpose of the REST API requests. The server processes the requests based on these given components and returns an appropriate response.
Here are some examples of using REST API, using the JSONPlaceholder API , which is an extremely basic mock API built for testing and learning purposes. Each endpoints below will send GET
requests to the API. Click to see API responses for requests.
Get the comments belonging the post with id 1
- Wikidata is a free and collaborative knowledge base that can be edited by anyone, serving as a central repository for structured data.
- The Wikidata API allows developers to access and interact with the data stored in Wikidata programmatically.
- The official documentation for the Wikidata API can be found on the Wikimedia website.
- The Wikibase Rest API endpoint is
https://wikidata.org/w/rest.php/wikibase/v0
.
- The API provides methods to retrieve information about items, properties, queries, and more.
- One can retrieve data in various formats such as JSON, XML, and others.
- SPARQL (SPARQL Protocol and RDF Query Language) is commonly used to query Wikidata. SPARQL queries can be executed using the API.
- The MediaWiki Action API provides a way to access and interact with MediaWiki-powered wikis programmatically.
- Designed to allow developers to perform various actions, retrieve data, and manipulate content on MediaWiki wikis.
- Official documentation for the MediaWiki Action API is available on the MediaWiki website.
- The API endpoint is typically
https://www.mediawiki.org/w/api.php
.
- The API supports a range of actions, including fetching page content, editing pages, searching, and more.
- Some actions may require authentication, and OAuth is commonly used for secure access.
- Wikidata is a free and open knowledge base that can be read and edited by both humans and machines. Wikidata acts as central storage for the structured data of its Wikimedia sister projects including Wikipedia, Wikivoyage, Wiktionary, Wikisource, and others.
- Loosely, it can be described Wikidata as Wikipedias database with over 46 million data items (April 2018).Wikidata also provides support to many other sites and services beyond just Wikimedia projects.
Like Wikipedia, there are all kinds of data stored in Wikidata. As such, when you are looking for a specific dataset or if you want to answer a curious question, it can be a good start looking for that data at Wikidata first.
Example questions:
- What is the capital city of every member of the European Union and how many inhabitants live there?
- Which countries use 112 as an emergency number?
- A free and open knowledge base that can be read and edited by both humans and machines.
- It contains various data types (e.g. text, images, quantities, coordinates, geographic shapes, dates).
- It uses SPARQL.
- SPARQL is a query language for RDF databases. In contrast to relational databases like SQL, items are not part of any tables. Instead, items are linked with each other like a graph or network:
To describe these relations, we can use a triple(A triple is a statement containing a subject predicate and object):
- Germany (subject) has the capital (predicate) Berlin (object).
- Berlin (subject) has the coordinates (predicate) 3.5million (object).
- The European Union (subject) has the member (predicate) Germany (object).
- Germany (subject) is a member of (predicate) the European Union (object).
If you want to dive deeper into the concept of SPARQL, I recommend this Youtube Video.
-
To get data from Wikidata you simply use triples (like to one above) to write a SPARQL query. Let’s have a look how such a SPARQL query might look like. Note, that we are using specific identifiers to define the right relationship and item:
SELECT ?country WHERE { ?country wdt:P463 wd:Q458. #country #member of #European Union }
-
Here, we simply ask for the countries that are part of the European Union.
-
Do you recognize the subject-predicate-object statement? We just select those countries, for which the condition holds: the country ( ?country ) is a member of (wdt:P463) the European Union (wd:Q458).
- Semantic search is a search engine technology that interprets the meaning of words and phrases. The results of a semantic search will return content matching the meaning of a query, as opposed to content that literally matches words in the query.
- When a query is launched, the search engine transforms the query into embeddings, which are numerical representations of data and related contexts. They are stored in vectors.
- The kNN algorithm, or k-nearest neighbor algorithm, then matches vectors of existing documents (a semantic search concerns text) to the query vectors.
- The semantic search then generates results and ranks them based on conceptual relevance.
- Semantic Mapping Semantic mapping is about visualizing relationships between concepts and entities (as well as relationships between related concepts and entities).
- Semantic Coding The idea behind using code to express meaning (not just presentation).
- H1-H6 subheadings would map out the main topics of a document
- Other HTML tags would express more semantics, including:
- If someone is searching for [pizza], they may be looking to:
- Order a pizza
- Cook a pizza
From years of serving search results to users and analyzing their interactions with those search results, Google seems to know that the majority of people searching for [pizza] are interested in ordering pizza.
- A built semantic search system to understand example to understand "how to" process of semantic search.
Put simply, semantic analysis is an attempt to bridge the gap between search algorithm, web pages it returns and the search engine’s users:
- A human being wants to find something and a search engine has two tasks to solve – Understand what the user wants and match that intent to web documents that do the best job meeting it
- A search engine needs to understand what it is they want to find. The semantic analysis is used to better understand the search query intent
- A search engine needs to match that query intent with web pages it has in the index. The semantic coding can be used to explain to a search engine what it is on the page and whether it matches the query intent.