Var keyword is just similar to variable declaration in Java whereas Val is little different. Once a variable is declared using Val the reference cannot be changed to point to another reference. This functionality of Val keyword in Scala can be related to the functionality of java final keyword. Val refers to immutable declaration of …
Difference between var and val in spark- Interview question Read More »
Lazy evaluation in Spark means that the execution will not start until an action is triggered. The Spark Lazy evaluation, users can divide into smaller operations. It reduces the number of passes on data by transformation grouping operation. By lazy evaluation in Spark to saves the trip between driver and cluster, speed up the process. …
Read CSV with spark 2.0 STEP 1. Open the spark-shell and fire the following command. scala> spark.read.format(“csv”).option(“header”,”true”).load(“F:/Hadoop Youtube/customer.csv”) STEP 2. Display the result with show command scala> .show +—–+——+———–+——-+———-+—-+——+ |empno| ename|designation|manager| hire_date| sal|deptno| +—–+——+———–+——-+———-+—-+——+ | 7369| SMITH| CLERK| 7902|12/17/1980| 800| 20| | 7499| ALLEN| SALESMAN| 7698| 2/20/1981|1600| 30| | 7521| WARD| SALESMAN| 7698| 2/22/1981|1250| 30| …
Updated: Dec 12, 2018 #hadoop #sqoop #defaultmapper #defaultreducer #hadoopinterviewquestion In this post we are going to focus the default number of mappers and reducers in the sqoop. scope is the part of Hadoop ecosystem which is mainly useful to move the data from the RDBMS database to hdfs file system or to directly hive tables and …
Default Number of mapper and reducer in SQOOP job Read More »
This article gives the difference between hadoop and big data. I personally found many students have confusion between hadoop and big data. Actually both are different entity. In one sentense I would say “big data is the problem” and “Hadoop is framework which provide the solution to resolve big data problem.” When you go for …
I have seen a many people those who want to switch their career from the Oracle DBA to the Hadoop but they are not sure how to start and from where to start it. after working so many years in the Oracle DBA Technology recently I have sw itch my career from the Oracle DBA …
Switch your career from Oracle DBA to Hadoop Bigdata Read More »