Good Friends are like Stars

Big Data & Machine learning

   
       Today we were learning  about Big Data & Machine learning. Mr Uthayasangar (lecturer in Moraduwa university)  taught about communication, machine learning & Big Datas.

💥Machine learning

             Machine learning focuses on the development of computer programs that can change when exposed to new data. The process of machine learning is similar to that of data mining.In machine learning, computers apply statistical learning techniques to automatically identify patterns in data. These techniques can be used to make highly accurate predictions. Keep scrolling.

 💥What are the different types of machine learning?

     These algorithms can be applied to almost any data problem:

• Linear Regression.
• Logistic Regression.
• Decision Tree.
• SVM.
• Naive Bayes.
• KNN.
• K-Means.
• Random Forest.

    💥categorization of machine learning tasks arises when one considers the desired output of a machine-learned system

• In classification inputs are divided into two or more classes, and the learner must produce a model that assigns unseen inputs to one or more  of these classes. This is typically tackled in a supervised way. Spam filtering is an example of classification, where the inputs are email (or other) messages and the classes are "spam" and "not spam".

• In regression also a supervised problem, the outputs are continuous rather than discrete.

• In clustering a set of inputs is to be divided into groups. Unlike in classification, the groups are not known beforehand, making this typically an unsupervised task.

💥What is meant by machine learning algorithms?

   Evolved from the study of pattern recognition and computational learning theory in artificial intelligence, machine learning explores the study and construction of algorithms that can learn from and make predictions on data – such algorithms overcome following strictly static program instructions by making data-driven .

💥 BIG DATA

Data which are very large in size is called Big Data. Normally we work on data of size MB(WordDoc ,Excel) or maximum GB(Movies, Codes) but data in Peta bytes i.e. 10^15 byte size is called Big Data.

💥From where this data comes from

These data come from many sources like

• Social networking sites: Facebook, Google, LinkedIn all these sites generates huge amount of data on a day to day basis as they have billions of users worldwide.

• E-commerce site: Sites like Amazon, Flipkart, Alibaba generates huge amount of logs from which users buying trends can be traced.

• Weather Station: All the weather station and satellite gives very huge data which are stored and manipulated to forecast weather.

• Telecom company: Telecom giants like Airtel, Vodafone study the user trends and accordingly publish their plans and for this they store the data of its million users.

• Share Market: Stock exchange across the world generates huge amount of data through its daily transaction.

      💥   Hadoop is an open source framework. It is provided by Apache to process and analyze very huge volume of data. It is written in Java and currently used by Google, Facebook, LinkedIn, Yahoo, Twitter etc.

💥Communication

Communication is the act of expressing (or transmitting) ideas, information, knowledge, thoughts, and feelings, as well as understanding what is expressed by others.

💥How to Develop Good Communication Skills

  

💥How do you communicate well with others?

1. Pause before responding. 
2. Be trustworthy and honest. 
3. Don't rush communication.
4. Adapt your ideas to others. 
5. Stay in the moment. 
6. Pay attention to non-verbal cues. 
7. Intend to understand. 
8. Be patient and open-minded.

HTTP Method

          The number of HTTP methods we'll use is quite small—there are just four HTTP "verbs" we'll need to know! They are:

💢 GET: retrieves information from the specified source (you just saw this one!)

💢 POST: sends new information to the specified source.

💢 PUT: updates existing information of the specified source.

💢 DELETE: removes existing information from the specified source.

💥Endpoints

      Endpoints are API-defined locations where particular data are stored. Just as we'll GET a pair of pants from PantsWorld or we'll GET a bag of peanuts from PeanutHut, we'll GET something different depending on the endpoint we use.

💥Authentication & API Keys

     Many APIs require an API key. Just as a real-world key allows we to access something, an API key grants we access to a particular API. Moreover, an API key identifies we to the API, which helps the API provider keep track of how our service is used and prevent unauthorized or malicious activity.
Some APIs require authentication using a protocol called OAuth. We won't get into the details, but if we've ever been redirected to a page asking for permission to link an application with our account, we've probably used OAuth.

HTTP

The Hypertext Transfer Protocol (HTTP) is an application protocol for distributed, collaborative, and hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web. Hypertext is structured text that uses logical links (hyperlinks) between nodes containing text.

 💥Why we have to use HTTP?

HTTPS is used to protect transmitted data from eavesdropping. It is the default protocol for conducting financial transactions on the web, and can protect a website's users from censorship by a government or an ISP.

💥The Client/Server Relationship

            The Internet is full of clients who ask for various resources (web pages, files, and so on), and servers, who store that information (or know where to get it). When we make an HTTP request, it zips through the Internet until it finds the server that knows how to fulfill that request. Then the server sends a response back to we.

 💥REST
 client/server relationship is a prerequisite of a set of principles called REST (or Representational State Transfer).

💥RESTful

        Every major development language now includes frameworks for building RESTful Web services. As such, it is important for Web developers and architects to have a clear understanding of REST and RESTful services.
          A service based on REST is called a RESTful service. REST is not dependent on any protocol, but almost every RESTful service uses HTTP as its underlying protocol.

         A RESTful design promises that and more. In general, RESTful services should have following properties and features.

• Representations
• Messages
• URIs
• Uniform interface
• Stateless
• Links between resources
• Caching

 💥An API or web service to be RESTful, it must do the following: 

👱Separate the client from the server  

👱Not hold state between requests (meaning that all the information necessary to respond to a request is available in each individual request; no data, or state, is held by the server from request to request) 

👱Use HTTP and HTTP methods (as explained in the next section).



💥Request & Response

     Clients send a request to the server, and the server replies with a response. Apart from the actual data, these messages also contain some metadata about the message. It is important to have some background about the HTTP 1.1 request and response formats for designing RESTful Web services.

Example:
 

GET http://www.w3.org/Protocols/rfc2616/rfc2616.html HTTP/1.1 Host: www.w3.org Accept: text/html,application/xhtml+xml,application/xml; … User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 … Accept-Encoding: gzip,deflate,sdch Accept-Language: en-US,en;q=0.8,hi;q=0.6     HTTP request   <VERB> is one of the HTTP methods like GET, PUT, POST, DELETE, OPTIONS, etc  <URI> is the URI of the resource on which the operation is going to be performed  <HTTP Version> is the version of HTTP, generally "HTTP v1.1" . <Request Header> contains the metadata as a collection of key-value pairs of headers and their values. These settings contain information about the message and its sender like client type, the formats client supports, format type of the message body, cache settings for the response, and a lot more information.  <Request Body> is the actual message content. In a RESTful service, that's where the representations of resources sit in a message.  HTTP response HTTP/1.1 200 OK Date: Sat, 23 Aug 2014 18:31:04 GMT Server: Apache/2 Last-Modified: Wed, 01 Sep 2004 13:24:52 GMT Accept-Ranges: bytes Content-Length: 32859 Cache-Control: max-age=21600, must-revalidate Expires: Sun, 24 Aug 2014 00:31:04 GMT Content-Type: text/html; charset=iso-8859-1

 HTTP response:

The server returns <response code>, which contains the status of the request. This response code is generally the 3-degits HTTP status code.

<Response Header> contains the metadata and settings about the response message.

<Response Body> contains the representation if the request was successful.
 
    

API

 

Application Program Interface

 In computer progamming, an application programming interface (API) is a set of subrotuni definitions, protocols, and tools for building application software.

 

💥Why APIs are important for business

      The web, software designed exchange information via the internet and cloud computer have all combined to increase the interest in APIs in general and services in particular.

💥Three basic types of APIs

APIs take three basic forms:
   👱 local
   👱web-like 
   👱program-like

SASS Functions

Nested Rules

Sass allows CSS rules to be nested within one another. The inner rule then only applies within the outer rule’s selector. For example:

#main p {
  color: #00ff00;
  width: 97%;

  .redbox {
    background-color: #ff0000;
    color: #000000;
  }
}

It is compiled to CSS

#main p {
  color: #00ff00;
  width: 97%; }
  #main p .redbox {
    background-color: #ff0000;
    color: #000000; }

This helps avoid repetition of parent selectors, and makes complex CSS layouts with lots of nested selectors much simpler. 

For example:

#main {
  width: 97%;

  p, div {
    font-size: 2em;
    a { font-weight: bold; }
  }

  pre { font-size: 3em; }
}

it is compiled to CSS

#main {
  width: 97%; }
  #main p, #main div {
    font-size: 2em; }
    #main p a, #main div a {
      font-weight: bold; }
  #main pre {
    font-size: 3em; }

💥LIST

Lists are just a series of other values, separated by either spaces or 
commas. In fact, individual values count as lists, too: they’re just 
lists with one item. 

E.g:  1px 2px, 5px 6px or (1px 2px) (5px 6px) 

💥Maps

    Maps represent an association between keys and values, where keys are used to look up values.

 E.g:
       $map: (key1: value1, key2: value2, key3: value3); Unlike lists, maps must always be surrounded by parentheses and must always be comma-separated. Both the keys and values in maps can be any SassScript object. 

💥 Operations

All types support equality operations (== and !=). In addition, each type has its own operations that it has special support for.

💥Number Operations

SassScript supports the standard arithmetic operations on numbers (addition +, subtraction -, multiplication *, division /, and modulo %). 

💥Functions

    SassScript defines some useful functions that are called using the normal CSS function syntax:

p {
  color: hsl(0, 100%, 50%);
}

is compiled to:

p {
  color: #ff0000; }

💥 @import

     Sass extends the CSS @import rule to allow it to import SCSS and Sass files. All imported SCSS and Sass files will be merged together into a single CSS output file. 

In addition, any variables or mixins defined in imported files can be used in the main file.

 E.g:

@import "foo.scss";

or

@import "foo";

💥Partials

If we have a SCSS or Sass file that wewant to import but don’t want to compile to a CSS file, we can add an underscore to the beginning of the filename. 
This will tell Sass not to compile it to a normal CSS file. we can then import these files without using the underscore.
For example, we might have _colors.scss.

Thought have energy

 

Thought have energy. Make sure our thoughts are positive and powerful 😅

SASS syntax

     SASS supports two syntaxes namely SCSS and Indented syntax.

• The SCSS (Sassy CSS) is an extension of CSS syntax. This means every valid CSS is a valid SCSS as well. SCSS makes much easier to maintain large stylesheets and can recognize vendor specific syntax, Many CSS and SCSS files use the extension .scss.

• Indented − This is older syntax and sometimes just called as SASS. Using this form of syntax, CSS can be written concisely. SASS files use the extension .sass.
  
  💥How to write placeholder selector in SASS?
  
  
  SASS supports placeholder selector using class or id selector. In normal CSS, these are specified with "#" or ".", but in SASS they are replaced with "%".
  
  💥Mention the different types of operations on the SASS?
   
  

  There are 5 types of operations −
  

  • Number Operations
  • Color Operations
  • String Operations
  • Boolean Operations
  • List Operations
    
    💥What is import directive?
     
    
    It directly takes the filename to import and all the imported files will get combined in a single CSS file.
    

  
  

SASS

💥SASS (Syntactically Awesome Stylesheet) 

      It is a CSS pre-processor, which helps to reduce repetition with CSS and saves time. It is more stable and powerful CSS extension language that describes the style of document structurally. This tutorial covers the basics of SASS.

💥History

        It was initially designed by Hampton Catlin and developed by Natalie Weizenbaum in 2006. Later, Weizenbaum and Chris Eppstein used its initial version to extend the Sass with SassScript.

💥Why to Use SASS?

• It is a pre-processing language which provides indented syntax (its own syntax) for CSS.
  
• It provides some features, which are used for creating stylesheets that allows writing code more efficiently and is easy to maintain.
  
• It is a super set of CSS, which means it contains all the features of CSS and is an open source pre-processor, coded in Ruby.
  
• It provides the document style in a good, structured format than flat CSS. It uses re-usable methods, logic statements and some of the built-in functions such as color manipulation, mathematics and parameter lists.

💥 Features of SASS

• It is more stable, powerful, and compatible with versions of CSS.
  
• It is a super set of CSS and is based on JavaScript.
  
• It is known as syntactic sugar for CSS, which means it makes easier way for user to read or express the things more clearly.
  
• It uses its own syntax and compiles to readable CSS.
  
• You can easily write CSS in less code within less time.
  
• It is an open source pre-processor, which is interpreted into CSS.

  
💥Advantages of SASS

• It allows writing clean CSS in a programming construct.
  
• It helps in writing CSS quickly.
  
• It is a superset of CSS, which helps designers and developers work more efficiently and quickly.
  
• As Sass is compatible with all versions of CSS, we can use any available CSS libraries.
  
• It is possible to use nested syntax and useful functions such as color manipulation, mathematics and other values.

 💥Disadvantages of SASS

• It takes time for a developer to learn new features present in this pre-processor.
  
• If many people are working on the same site, then should use the same preprocessor. Some people use Sass and some people use CSS to edit the files directly. Therefore, it becomes difficult to work on the site.
  
• There are chances of losing benefits of browser's built-in element inspector.

JSON

.

 💥JavaScript Object Notation (JSON)

 JSON is a syntax for storing and exchanging data.

   💥What is JSON?

• JSON stands for JavaScript Object Notation
• JSON is a lightweight data-interchange format
• JSON is "self-describing" and easy to understand
• JSON is language independent *

     
 💥Why use JSON?

Since the JSON format is text only, it can easily be sent to and from a server, and used as a data format by any programming language.

 💥Exchanging Data

👉When exchanging data between a browser and a server, the data can only be text.

👉JSON is text, and we can convert any JavaScript object into JSON, and send JSON to the server.

👉We can also convert any JSON received from the server into JavaScript objects.

👉This way we can work with the data as JavaScript objects, with no complicated parsing and translations.

💥Convert a JavaScript object into a JSON string, and send it to the server(Sending Data)
 
     😅<script>
var myObj = { "name":"Abi", "age":11, "city":"Jaffna" };
var myJSON = JSON.stringify(myObj);
window.location = "demo_json.php?x=" + myJSON;
</script>



out put: Abi from Jaffna is 11

👉 Convert a string written in JSON format, into a JavaScript object.(Receiving Data)

      😅 <script>

var myJSON = '{ "name":"Abi", "age":31, "city":"New York" }';
var myObj = JSON.parse(myJSON);
document.getElementById("demo").innerHTML = myObj.name;

</script>

out put: Abi


💥JSON  Data types

     In JSON, values must be one of the following data types:

• a string   { "name":"John" } 

• a number  { "age":30 } 

• an object  {"employee":{ "name":"John", "age":30, "city":"New York" }
  } 

• an array  [ "Ford", "BMW", "Fiat" ] 

• a boolean { "sale":true }

• null  { "middlename":null }

 

  JSON values cannot be one of the following data types:

• a function
• a date
• undefined

 

   👉We can access the object values by using dot (.) or bracket ([ ])  notation

   <script>

var myObj, x;
myObj = { "name":"Sutha", "age":30, "car":null };
x = myObj.age;
document.getElementById("demo").innerHTML = x;

</script>

output: 30

   <script>

var myObj, x;
myObj = { "name":"Sutha", "age":30, "car":null };
x = myObj["name"];
document.getElementById("demo").innerHTML = x;

</script>

output: Sutha 


💥 Looping an Object

we can loop through object properties by using the for-in loop and also  use the bracket notation to access the property values

       😅<script>
var myObj = { "name":"John", "age":30, "car":null };
for (x in myObj) {
    document.getElementById("demo").innerHTML += x + "<br>";
}
</script>

output:   name
              age
              car

      😅<script>
var myObj = { "name":"John", "age":30, "car":null };
for (y in myObj) {
    document.getElementById("demo").innerHTML += myObj[y] + "<br>";
}
</script>

output:John
           30
           null
 

 💥Delete object

 

       😅<script>
var myObj, i, x = "";
myObj = {
  "name":"John",
  "age":30,
  "cars": {
    "car1":"Ford",
    "car2":"BMW",
    "car3":"Fiat"
  }
}
delete myObj.cars.car2;

for (i in myObj.cars) {
    x += myObj.cars[i] + "<br>";
}

document.getElementById("demo").innerHTML = x;

</script>

 

output:
              Ford
            Fiat

 

 

nodejs with mongodb

 

 

💥Database Created

To create a database in MongoDB, start by creating a MongoClient object, then specify a connection URL with the correct ip address and the name of the database you want to create.

💥Creating a Table

 To create a table in mongodb use the creteCollection()

💥 Insert Into Table

To insert a record into a table in MongoDbB, We use the insertOne().

 💥Select One

To select data from a table in MongoDB we can use the findOne()

 💥Node.js MongoDB Query

When selecting records from a table, you can filter the result by using a query object.
We can use find() method for limit the search

 💥Node.js MongoDB Remove

Delete Record

We can use remove() command to delete records or documents in Node.js MongoDB

💥Drop Collection

we use drop() methode for to delete a table or collection in node.js MongoDB 

var MongoClient = require('mongodb').MongoClient;
var url = "mongodb://localhost:27017/mydb";
MongoClient.connect(url, function(err, db) {
  if (err) throw err;
  db.dropCollection("customers", function(err, delOK) {
    if (err) throw err;
    if (delOK) console.log("Table deleted");
    db.close();
  });
}); 

 

 Save and run this file we get the message "table deleted"