apache spark - How can I change SparkContext.sparkUser() setting (in pyspark)? -


i new spark , pyspark.
use pyspark, after rdd processing, tried save hdfs using saveastextfile() function. 'permission denied' error message because pyspark tries write hdfs using local account, 'kjlee', not exist on hdfs system.

i can check spark user name sparkcontext().sparkuser(), can't find how change spark user name.

how can change spark user name?

there environment variable : hadoop_user_name use export hadoop_user_name=anyuser or in pyspark can use os.environ["hadoop_user_name"] = "anyuser"


Comments

Post a Comment

Popular posts from this blog

java - WARN : org.springframework.web.servlet.PageNotFound - No mapping found for HTTP request with URI [/board/] in DispatcherServlet with name 'appServlet' -

this question has answer here: why spring mvc respond 404 , report “no mapping found http request uri […] in dispatcherservlet”? 3 answers i trying making simple bbs i'm encountering following error, warn : org.springframework.web.servlet.pagenotfound - no mapping found http request uri [/board/] in dispatcherservlet name 'appservlet' i have tried solve error many times, have failed. how can solve error? here web.xml, controllerand , servlet-context. my web.xml <?xml version="1.0" encoding="utf-8"?> <web-app version="2.5" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:xsi="http://www.w3.org/2001/xmlschema-instance" xsi:schemalocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"> <filter> <filter-nam...
Read more

html - Outlook 2010 Anchor (url/address/link) -

i'm trying link specific pdf page msoutlook'10 or msword'10, #page=... anchor particularly useful , works fine elsewhere. unfortunately when click [ctr-k] in outlook or winword, links re-formatted assuming unc path. pdf file located on shared drive. using "file:" prefix links open pdf in adobe reader (outside of browser), , "http:" open inside browser. issue seem need open in browser "file:" prefix in order anchor work. know anchors work inside browser. outlook uses windows/explorer default program (based on file extension) seems open in adobe reader neglecting anchor. is aware of solution, link specific page or bookmark of shared-pdf ms outlook and/or winword link? common references: mailermailer.com/...anchor-tags-html-emails campaignmonitor.com/...anchor-links-in-email-newsletters/ stackoverflow.com/...anchor-in-pdf-document
Read more

android - How to create dynamically Fragment pager adapter -

i unable create dynamic layout each tabs, able create tabs dynamically constructor(public mypageradapter(fragmentmanager fm , int nooftabs)), coudn't inflate view each tabs creating each fragment should dyanmic. public class mypageradapter extends fragmentpageradapter { private final string[] titles = { "categories", "home", "top paid", "top free", "top grossing", "top new paid", "top new free", "trending" }; int nooftabs; **public mypageradapter(fragmentmanager fm , int nooftabs) { super(fm); this.nooftabs = nooftabs; }** /*public mypageradapter(fragmentmanager fm) { super(fm); }*/ @override public charsequence getpagetitle(int position) { return titles[position]; } @override public int getcount() { ...
Read more