Hadoop Installation



If you are working with Windows Then Follow These Steps Else If(Linux) you can skip to hadoop installation.


Download VMWare: VMWare


Properties To Be Modified As Following:


RAM-4GB

ROM(MEMORY)-30GB

CORES-4


ISO FILE LINK: https://ubuntu.com/download/desktop/thank-you?version=22.04.2&architecture=amd64


Accept all permissions and start using Linux.



Hadoop Installation:Hadoop Installation Steps

Comments

Post a Comment

Popular posts from this blog

HDFS(Hadoop Distributed File System)

Image
  In which platform Hadoop can be implemented ? Windows Mac Linux Working and principles of HDFS Information About the data is transferred to the Master Node(JP1) It's not data transfer Client API acts intermediate between Master and Slave. The information transferred to Master is operation, filename, size, etc. The Acknowledgement is sent to the Master from Slave about the requested operation. Standalone file system is needed to create distributed file system. Through Edge node(Our PC's) we will be sending data to Slave node and the edge node doesn't contain any HDFS data as we know that data never comes back from HDFS. When an request is sent to slave node then Client API sends information to the master then it sends the write/read operation information to MetaData(Placement Allocation(Rack awareness Allocation), Block size, Replication) Pipeline is used to distribute data. The client API is not only responsible for interaction between Master and Slave. The Master and Sla...
Read more

NOSQL Database and Pipeline tool

Image
  Kafka is a data pipeline tool. Zookeeper is needed before installing Kafka as its a cluster coordinator. Kafka stores all metadata information in zookeeper. NOSQL CAP is not possible at a time in NOSQL databases. Products like Facebook, Instagram achieves CAP by using multiple databases. CA is not possible at a time because of partition. In RDBMS no replication is present so CA is achieved. Ex: Hbase, MongoDB, Cassandra Datastores Keyvalue Datastore(Reddis) ColumnOriented Datastore(Hbase, Cassandra) DocumentOriented Datastore(MongoDB) Graph Datastore(Neo4j) Hbase is used for internal operations Lookup(veryfast) upset(insert+update)
Read more

Spark Standalone Architecture and Working

Image
  About Spark Standalone Architecture The Daemons in spark are(Master Slave Architecture): Master Node Worker Node The Driver Program is starting point of spark which is started by Master and Responsible for Job. Lets understand the architecture of spark with an Example Client submitted job in the form of jar Each jar file contains driver program. The Spark master executes the driver code. The driver is requesting for resource from cluster manager. The cluster manager is responsible for allocation of resources. The cluster manager is allocating resources in the form of executors. The cluster manager finds data locality(particular data location) before assigning executors. Driver program is responsible for assigning the job to the executors (JAR File). Once the executor has been created it starts sending heartbeat to driver program and cluster manager also informs the driver program that it has been allocated the resources. The driver program is in Master and Executors are Workers. ...
Read more