hadoop - Oozie with Spark Java action -


i have tried oozie spark using java action.

  • oozie version 4.1.0.3
  • spark version 1.2.1.

the same code working fine when run using spark-submit. oozie, giving exception.

in java code, have selection statement using hivecontext.sal("select * from.."), @ time of getting collection.

note: have no found 1s in hive-site.xml

exception:

java.lang.runtimeexception: java.lang.numberformatexception: input string: "1s"

find workflow.xml used below

<action name="scala_java">     <java>     <job-tracker>${jobtracker}</job-tracker>     <name-node>${namenode}</name-node>     <job-xml> ${namenode}/user/${wf:user()}/${approot}/mysite/hive-site.xml </job-xml>     <configuration>     <property>     <name>oozie.hive.defaults</name>     <value>${namenode}/user/${wf:user()}/${approot}/mysite/hive-default.xml</value>     </property>     <property>     <name>pool.name</name>                                                     <value>${etlpoolname}</value>     </property>     <property>                                                  <name>oozie.use.system.libpath</name>                                                  <value>true</value>     </property>     <property>                                                    <name>oozie.launcher.mapreduce.job.user.classpath.first</name>     <value>true</value>     </property>     <property>                                                    <name>oozie.launcher.pool.name</name>                                                  <value>${oozielauncherpoolname}</value>     </property>      <property>                                                 <name>datanucleus.autocreateschema</name>                                                 <value>false</value>     </property>     <property>                                               <name>datanucleus.fixeddatastore</name>                                   </property>     </configuration>     <main-class>org.apache.spark.deploy.sparksubmit</main-class>         <arg>--master</arg>         <arg>yarn</arg>         <arg>--num-executors</arg>         <arg>64</arg>         <arg>--executor-memory</arg>         <arg>8g</arg>         <arg>--driver-memory</arg>         <arg>2g</arg>         <arg>--executor-cores</arg>         <arg>5</arg>         <arg>--queue</arg>         <arg>testq</arg>                                                <arg>--class</arg>         <arg>test</arg>         <arg>myscala-0.0.1-snapshot.jar</arg>         <file>myscala-0.0.1-snapshot.jar</file>        </java>     </action>

how solve problem?


Comments

Post a Comment

Popular posts from this blog

1111. appearing after print sequence - php -

morning, i have code takes string of shoe sizes, explodes , puts in table. code works , info displayed correctly on page after table there few 1's after table. i'm confused this. so table looks (in columns of 3 on webpage) available in following sizes: 3 4 5 6 9 10 111 here's codes, purpose of example $x = 3|4|6|10|9 function get_sizes_pipe($x) { $counter = 0; $splits = explode("|",$x); asort($splits); $x = print "<table class='greentable'>"; $x .= print "<thead><tr><th scope='col' colspan='3' abbr='starter'>available in following sizes:</th></tr></thead><tbody><tr>"; foreach ($splits $split) { $entry = print "<td>".$split."</td>"; if($counter == 2){ $entry .= print "</tr><tr>"; $counter =0; } else { $counter++; ...
Read more

java - WARN : org.springframework.web.servlet.PageNotFound - No mapping found for HTTP request with URI [/board/] in DispatcherServlet with name 'appServlet' -

this question has answer here: why spring mvc respond 404 , report “no mapping found http request uri […] in dispatcherservlet”? 3 answers i trying making simple bbs i'm encountering following error, warn : org.springframework.web.servlet.pagenotfound - no mapping found http request uri [/board/] in dispatcherservlet name 'appservlet' i have tried solve error many times, have failed. how can solve error? here web.xml, controllerand , servlet-context. my web.xml <?xml version="1.0" encoding="utf-8"?> <web-app version="2.5" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xsi:schemalocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"> <filter> <filter-nam...
Read more

Ruby on Rails, ActiveRecord, Postgres, UTF-8 and ASCII-8BIT encodings -

weirdest error, who’s challenge try , me? spent hours yesterday on , it’s magic. can't believe happening , it's driving me crazy. btw, using: rails 4.2.3 ruby 2.2.3 pg 0.18 encoding.default_internal = encoding.default_external = utf-8 config.encoding = 'utf-8' it has postgres, activerecord , encodings! turns out we’ve been getting these errors whenever create new users on database special characters: encoding::undefinedconversionerror: "\xc3" ascii-8bit utf-8 \xc3 may vary depending on characters. weird part we’ve set since beginning utf-8. makes no sense , digged it, , used script: user.all.each |user| user.attributes.each |name, value| if value.is_a? string puts user.email + name.encoding.to_s + value.encoding.to_s end end end and output of 1 of users, users same: dr_lottahelp@blahblah.comutf-8utf-8 dr_lottahelp@blahblah.comutf-8ascii-8bit dr_lottahelp@blahblah.comutf-8ascii-8bit dr_lottahelp@blahblah.comutf-8asci...
Read more