Hive shell is not starting up

lalitha@lalitha-VirtualBox:~/Downloads/apache-hive-0.13.0-bin$ hive
Logging initialized using configuration in jar:file:/home/lalith/Downloads/apache-hive-0.13.0-bin/lib/hive-common-0.13.0.jar!/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException:java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient


I am running a single node hadoop cluster on ubuntu.
I have seen many answers upon this question, I tried all of them, still the error persists.

hive-site.xml:

<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:derby:;databaseName=/home/lalitha/Downloads/apache-hive-2.1.0-bin/metastore_db;create=true</value>
</property>
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/hive/warehouse</value>
</property>
<property>
<name>hive.metastore.uris</name>
<value/>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>org.apache.derby.jdbc.EmbeddedDriver</value>
</property>
<property>
<name>javax.jdo.PersistenceManagerFactoryClass</name>
<value>org.datanucleus.api.jdo.JDOPersistenceManagerFactory</value>
</property>
</configuration>


Could anyone please help me.

EDIT:
Following is the output after running:
hive -hiveconf hive.root.logger=DEBUG,console

lalitha@lalitha-VirtualBox:~/Downloads/apache-hive-0.13.0-bin$ hive -hiveconf hive.root.logger=DEBUG,console
18/06/30 13:34:48 [main]: DEBUG common.LogUtils: Using hive-site.xml found on CLASSPATH at /home/lalitha/Downloads/apache-hive-0.13.0-bin/conf/hive-site.xml

Logging initialized using configuration in jar:file:/home/lalitha/Downloads/apache-hive-0.13.0-bin/lib/hive-common-0.13.0.jar!/hive-log4j.properties
18/06/30 13:34:48 [main]: INFO SessionState:
Logging initialized using configuration in jar:file:/home/lalitha/Downloads/apache-hive-0.13.0-bin/lib/hive-common-0.13.0.jar!/hive-log4j.properties
18/06/30 13:34:48 [main]: DEBUG parse.VariableSubstitution: Substitution is on: hive
18/06/30 13:34:49 [main]: INFO metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.storeManagerType value null from jpox.properties with rdbms
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.option.NonTransactionalRead value null from jpox.properties with true
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.autoStartMechanismMode value null from jpox.properties with checked
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.transactionIsolation value null from jpox.properties with read-committed
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.fixedDatastore value null from jpox.properties with false
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.validateColumns value null from jpox.properties with false
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.option.ConnectionURL value null from jpox.properties with jdbc:derby:;databaseName=/home/lalitha/Downloads/apache-hive-2.1.0-bin/metastore_db;create=true
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.validateConstraints value null from jpox.properties with false
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.identifierFactory value null from jpox.properties with datanucleus1
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.PersistenceManagerFactoryClass value null from jpox.properties with org.datanucleus.api.jdo.JDOPersistenceManagerFactory
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.cache.level2 value null from jpox.properties with false
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.rdbms.useLegacyNativeValueStrategy value null from jpox.properties with true
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding hive.metastore.integral.jdo.pushdown value null from jpox.properties with false
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.option.DetachAllOnCommit value null from jpox.properties with true
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.option.ConnectionDriverName value null from jpox.properties with org.apache.derby.jdbc.EmbeddedDriver
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.option.Multithreaded value null from jpox.properties with true
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.plugin.pluginRegistryBundleCheck value null from jpox.properties with LOG
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.validateTables value null from jpox.properties with fals
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.autoCreateSchema value null from jpox.properties with true
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.cache.level2.type value null from jpox.properties with none
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding datanucleus.connectionPoolingType value null from jpox.properties with BONECP
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: Overriding javax.jdo.option.ConnectionUserName value null from jpox.properties with APP
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.rdbms.useLegacyNativeValueStrategy = true
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: hive.metastore.integral.jdo.pushdown = false
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.autoStartMechanismMode = checked
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: javax.jdo.option.Multithreaded = true
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.identifierFactory = datanucleus1
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.transactionIsolation = read-committed
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.validateTables = false
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: javax.jdo.option.ConnectionURL = jdbc:derby:;databaseName=/home/lalitha/Downloads/apache-hive-2.1.0-bin/metastore_db;create=true
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: javax.jdo.option.DetachAllOnCommit = true
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: javax.jdo.option.NonTransactionalRead = true
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.fixedDatastore = false
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.validateConstraints = false
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: javax.jdo.option.ConnectionDriverName = org.apache.derby.jdbc.EmbeddedDriver
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: javax.jdo.option.ConnectionUserName = APP
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.validateColumns = false
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.cache.level2 = false
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.plugin.pluginRegistryBundleCheck = LOG
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.cache.level2.type = none
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: javax.jdo.PersistenceManagerFactoryClass = org.datanucleus.api.jdo.JDOPersistenceManagerFactory
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.autoCreateSchema = true
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.storeManagerType = rdbms
18/06/30 13:34:49 [main]: DEBUG metastore.ObjectStore: datanucleus.connectionPoolingType = BONECP
18/06/30 13:34:49 [main]: INFO metastore.ObjectStore: ObjectStore, initialize called
18/06/30 13:34:51 [main]: DEBUG bonecp.BoneCPDataSource: JDBC URL = jdbc:derby:;databaseName=/home/lalitha/Downloads/apache-hive-2.1.0-bin/metastore_db;create=true, Username = APP, partitions = 1, max (per partition) = 10, min (per partition) = 0, idle max age = 60 min, idle test period = 240 min, strategy = DEFAULT
18/06/30 13:34:52 [main]: ERROR Datastore.Schema: Failed initialising database.
Unable to open a test connection to the given database. JDBC url = jdbc:derby:;databaseName=/home/lalitha/Downloads/apache-hive-2.1.0-bin/metastore_db;create=true, username = APP.
Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------
java.sql.SQLException: Failed to start database '/home/lalitha/Downloads/apache-hive-2.1.0-bin/metastore_db' with class loader sun.misc.Launcher$AppClassLoader@154617c, see the next exception for details.
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
at org.apache.derby.jdbc.Driver20.connect(Unknown Source)
at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:208)
at com.jolbox.bonecp.BoneCP.obtainRawInternalConnection(BoneCP.java:361)
at com.jolbox.bonecp.BoneCP.<init>(BoneCP.java:416)
at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:120)
at org.datanucleus.store.rdbms.ConnectionFactoryImpl$ManagedConnectionImpl.getConnection(ConnectionFactoryImpl.java:501)
at org.datanucleus.store.rdbms.RDBMSStoreManager.<init>(RDBMSStoreManager.java:298)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:631)
at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:301)
at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:1187)
at org.datanucleus.NucleusContext.initialise(NucleusContext.java:356)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:775)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333)
at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965)
at java.security.AccessController.doPrivileged(Native Method)
at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960)
at javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808)
at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701)
at org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:309)
at org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:338)
at org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:247)
at org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:222)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:62)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.hive.metastore.RawStoreProxy.<init>(RawStoreProxy.java:58)
at org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:67)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore.java:498)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:476)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:524)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:398)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.<init>(HiveMetaStore.java:357)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:54)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:59)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newHMSHandler(HiveMetaStore.java:4948)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:171)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1410)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:62)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:72)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:2444)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:2456)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:338)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
Caused by: java.sql.SQLException: Failed to start database '/home/lalitha/Downloads/apache-hive-2.1.0-bin/metastore_db' with class loader sun.misc.Launcher$AppClassLoader@154617c, see the next exception for details.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
... 72 more
Caused by: java.sql.SQLException: Another instance of Derby may have already booted the database /home/lalitha/Downloads/apache-hive-2.1.0-bin/metastore_db.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
... 69 more
Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database /home/lalitha/Downloads/apache-hive-2.1.0-bin/metastore_db.


Still a lot more errors are there.
I am a new bie to hadoop and could not find what was the mistake and what was happening here.

1 Answer
1

Could you try running the following command and paste the output here?

hive -hiveconf hive.root.logger=DEBUG,console


Edit -
The output says
"Another instance of Derby may have already booted the database
/home/lalitha/Downloads/apache-hive-2.1.0-bin/metastore_db."
This seems to be a problem. Do a "ps -ef | grep hive" and kill all the processes and retry executing the command.

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.