base64 - OutOfMemory Exception while executing JAVA Mapping -


i have transfer big files (500 mb+...can of 1gb in size). these files have base64 encoded , encoded string has put in xml file. while below code works smaller files (30 - 50 mb) fails files great 100 mb. using base64 encoder sun (sun.misc.base64encoder).

public void execute(inputstream inputstream, outputstream outputstream) throws streamtransformationexception{         try         {             string sourcefilename = "test_file";             string receiverstr = "";             //2. convert input data in base64encoded string             base64encoder encoder = new base64encoder();             byte input[] = new byte[inputstream.available()];              inputstream.read(input);             string base64encoded = encoder.encode(input);             //3. build soap request format             string serverurl = "http://website/url";              string soapenvelope = "<soapenv:envelope xmlns:soapenv=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:soap=\"http://schemas.microsoft.com/sharepoint/soap/\">";             string soapheader = "<soapenv:header/><soapenv:body><soap:copyintoitems><soap:sourceurl>c:\\users\\desktop\\test_file.txt</soap:sourceurl><soap:destinationurls><soap:string>" + serverurl + "</soap:string></soap:destinationurls><soap:fields><soap:fieldinformation " + "type=" + "\"text\"" + " displayname=\"" + sourcefilename + "\"" + " internalname=\"" + sourcefilename + "\"" + " id=\"deff4b5c-b727-414c-893d-c56a8e12455f\"" + " value=\"" + sourcefilename + "\"/></soap:fields>";             string soapstream = "<soap:stream>" + base64encoded + "</soap:stream>";             receiverstr = soapenvelope + soapheader + soapstream + "</soap:copyintoitems></soapenv:body></soapenv:envelope>";             //4. write soap request receiver channel             outputstream.write(receiverstr.getbytes());         }         catch(exception e) {             throw new streamtransformationexception(e.tostring());           }     }

when try see message @ run-time, entire message not displayed , truncated in-between in base64encoded string. below error seen in system on executing java code. enter image description here please note server settings can otherwise transfer 1gb+ files without java heap size error or file truncation. can please let me know how can process big files using above logic?

thanks,

abhishek.

taking @ code store data 3 times:

the first time in byte array input stream. second time in encoded string , write output stream.

what can optimize bit split reading input stream/encoding , writing of encoded string output stream. way can let go of input byte array once encoded , memory can freed. better solution when encoder directly writes output stream or directly write encoding output stream.

however still have consider maximum file size should can processed , adjust heap setting accordingly: how deal "java.lang.outofmemoryerror: java heap space" error (64mb heap size)


Comments

Post a Comment

Popular posts from this blog

html - Outlook 2010 Anchor (url/address/link) -

i'm trying link specific pdf page msoutlook'10 or msword'10, #page=... anchor particularly useful , works fine elsewhere. unfortunately when click [ctr-k] in outlook or winword, links re-formatted assuming unc path. pdf file located on shared drive. using "file:" prefix links open pdf in adobe reader (outside of browser), , "http:" open inside browser. issue seem need open in browser "file:" prefix in order anchor work. know anchors work inside browser. outlook uses windows/explorer default program (based on file extension) seems open in adobe reader neglecting anchor. is aware of solution, link specific page or bookmark of shared-pdf ms outlook and/or winword link? common references: mailermailer.com/...anchor-tags-html-emails campaignmonitor.com/...anchor-links-in-email-newsletters/ stackoverflow.com/...anchor-in-pdf-document
Read more

javascript - Why does running this loop 9 times take 100x longer than running it 8 times? -

consider code: test = function() { } t = new test(); (var = 0; < 8; i++) { result = t instanceof test; } if change number of iterations 8 9 , loop take 100 times longer complete in latest version of firefox (41.0.1). tested on 2 different pcs , magic limit 8. here jsperf test used: http://jsperf.com/instanceof-8-times-vs-9-times does have idea why might happen? seems specific instanceof . not happen if else object, example check property. note: filed bugzilla bug this. jan de mooij mozilla team has posted details in bugzilla thread . here's simplistic interpretation of highly technical answers: in i < 8 case, firefox smart enough hoist result = t instanceof test; statement out of loop (according tests, doesn't seem omit altogether). in i < 9 case, apparently doesn't optimization . why? reason not clear, related fact 9 iterations threshold above function considered "hot" enough run through jit compiler. i < 8 cas...
Read more

Getting gateway time-out Rails app with Nginx + Puma running on Digital Ocean -

following this guide setup app on digital ocean droplet, not working. site not loading , it's throwing: "gateway time-out". the puma server works, since ps aux | grep puma prints: root 15878 0.0 0.4 376832 2168 ? sl 12:50 0:00 puma 2.13.4 (unix:///var/www/myapp/shared/tmp/sockets/myapp-puma.sock) [20150930164940] root 15881 0.0 0.7 903676 3692 ? sl 12:50 0:06 puma: cluster worker 0: 15878 [20150930164940] root 24654 0.0 0.1 11748 916 pts/2 s+ 18:00 0:00 grep --color=auto puma my nginx config: user root; worker_processes 1; events { worker_connections 1024; } http { upstream puma { server unix:///var/www/myapp/shared/tmp/sockets/myapp-puma.sock; } server { listen 80; server_name myapp.com; root /var/www/myapp/current/public; access_log /var/www/myapp/current/log/access.log; error_log /...
Read more