Split Byte Array Java [on hold] - java

How to split Byte array smaller chunks and encode to Base64 .
And how to send large amount (nearly more than 1GB) of encoded string in pieces by pieces through ajax to avoid transfer all the encoded string at a time.
Can anyone help?
i am using like this to encode huge byte array Base64.getEncoder().encodeToString(bytes); but getting heap space error.

Related

Java saving huffman encoded text to file

I'm working on a huffman compression and decompression application in java. So far I got the encoding and decoding working. It converts a big input text to it's encoded binary text. This is a String of 1's and 0's. For example:
String originaltext = "Hello I am trying to program a huffman application and..."
String encodedtext = "1100001110001111011010100110100110...." It's a pretty long string.
Now I want to save the string to a file as binary file to reduce the size. But when I try do this, the size will be way bigger then the original text size. Instead I need the size smaller then the original file size. After saving the encodedtext to a file I need to read the binary file back in and convert it to the encodedText string to deconvert it with my huffmantree method.
How can I save the binary string to a binary file which size is then smaller then the original size? And how do read the file in and convert the binary code to the encodedString text?
Probably you're writing your string of 1 and 0 as a string, which will result in 1 byte per 1 or 0.
You need to convert those 1 and 0 to bytes (i.e. convert groups of eight 1 or 0 into 1 byte and write those bytes.
EDIT
See answer by 6502 to this question for some code to convert the 1s and 0s to bytes.

Combining multiple image in to single file

To convert all images in a folder to a single file , i used \n as delimiter to separate images in that file. is this is good enough?. (If the image bytes contain \n bytes then it will become an issue while processing), is there any chance that an image file may contain \n's equivalent byte inside it.
Convert both images to byte arrays and then using base64 encoder, convert each byte array to string. Then you can use \n as delimiter between these two images.
To get the images back as byte arrays, you should decode those strings in the file using base64 decoder. I think almost all languages support Base64 encoding and decoding.

Why is my java String shorter in length than the byte[] array it was generated from?

I'm reading a blob from a MySql database using JDBC. I know the resulting byte array is good, I've sent it over HTTP as a string literal of numbers for each byte, and successfully downloaded the result (jpg). (just to prove mysql -> java servlet data is good).
Constructing a new string from this byte array using UTF-8 yields a string shorter in length than the byte array, and of values I can't decipher. If UTF-8 is AT LEAST 1 byte per character, shouldn't the resulting string be AT A MINIMUM the length of the byte array its generated from? (for this particular example, byte length is 12,079,474 and resulting string length is 11,501,845)
Thanks for your time!
In your bytes, you have data that is interpreted as continuation bytes, i.e. in UTF-8 they have special meaning and they form one Unicode character from multiple bytes. That is why your string is shorter than the number of bytes.

Gunzip/gzip compression ratio?

Situation:
I have some string that can be at most 240,000 characters long. I store this string in AWS DynamoDB. Since DynamoDB uses UTF-8 encoding and each String can be at most 400 KB, worst case scenario my string would be 240,000 char * 4 bytes/char = 960,000 bytes, which exceeds the limit of 409,600 bytes.
I did some testing on compressing strings. Based on the result, it seems like 1 byte UTF-8 character strings can be compressed by 25% using gzip and 2-3 byte UTF-8 character strings can be compressed by 60% using gzip.
I used this link to compress string to binary: http://www.txtwizard.net/compression
Question:
How much can I compress using gzip? What's the worst case and average case? (How much it can compress for 4 byte UTF-8 strings? 1 byte UTF-8 strings?)
If I have this 240,000 characters long string, can I always compress it to less than 400,000 bytes?

How to convert byte array of an image to a base64 encoded string representing a jpg

I am writing an Android app that takes photos and uploads them.
The API requires a base 64 encoded string representing the jpg to upload.
The Camera API gives me a byte [].
How can I go about converting this byte array into a base64 encoded string, ensuring I keep jpg format?
Currently, I am trying String encoded = Base64.encodeToString(byteArray, Base64.NO_WRAP);
But this somehow is invalid, giving me a bunch of AAAAAAAAAs in the log rather than a real base64 encoded string.
Thanks for any help
EDIT:
Actually, when I log out the contents of encoded above, I get
/9j/4UVIRXhpZgAATU0AKgAAAAgACQEPAAIAAAAPAAAAegEQAAIAAAAJAAAAigESAAMAAAABAAEAAAEaAAUAAAABAAAAlAEbAAUAAAABAAAAnAEoAAMAAAABAAIAAAITAAMAAAABAAEAAIdpAAQAAAABAAAApIglAAQAAAABAAAGCAAABi5MRyBFbGVjdHJvbmljcwAAVlM5ODUgNEcAAAAAAEgAAAABAAAASAAAAAEAFIKaAAUAAAABAAABmoKdAAUAAAABAAABoognAAMAAAABAGQAAJAAAAcAAAAEMDIyMJADAAIAAAAUAAABqpAEAAIAAAAUAAABvpEBAAcAAAAEAQIDAJIEAAoAAAABAAAB0pIHAAMAAAABAAIAAJIJAAMAAAABAAAAAJIKAAUAAAABAAAB2pJ8AAcAAAIAAAAB4pKGAAcAAAIAAAAD4qAAAAcAAAAEMDEwMKABAAMAAAABAAEAAKACAAQAAAABAAAFAKADAAQAAAABAAADwKAFAAQAAAABAAAF6qQDAAMAAAABAAAAAKQEAAUAAAABAAAF4gAAAAAAAAABAAAAGAAAAMgAAABkMjAxNTowNjowNSAyMjoxMzoxOAAyMDE1OjA2OjA1IDIyOjEzOjE4AAAAAAAAAAABAAAHCAAAA+hBTENTSUlGMFUAAAAAAAAAAQAPJ3ECAA0UIBAHEhUAAElJRjEAAAAAAABsAOsAAACdANAATACD/w0AcAAbAHUBmf/U/yUAaQA3ATL//XULZK54T27mdYBtWXreaK56Z26XceFoWWIZVx94k3BJd19u2VIoTgNW8lfWU4NWklYvU5dXjFhlUI1R20b4QuFV/1n2VKhXAAAAAA0AAAAAAAAAAAA5AAAAAAAAAAAASUlGNAAAAAAAAAAAAAAAAABw0QAAAAAAAAAAAAAAAAAAAAAAAQkREgAJAQkACQoB7u4AAgEXARgxBAEJERIACQoB//9VSUY0AIACAAAAAAC1BAAAEg0AAFxJVGgAAAAAtQQAABINAABcSQ9oAAAAAGZ3AABJSUY1QldMQapsuXcCa2JwB48AAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA6wAAAAAAAAAAAAAAAAABAAAAAQAAAAEAVUlGNQABABMAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAQBVSUY2QUxIQxeFAQDiGwAA/////wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAElJRjkAAAAAAAD/AAAAAAAAAP8AAAAAAEFMQ0UAVjQuNFJNMSAgICBGb2N1c0FyZWE9MDAwMDAwMDAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAZAAAAGQAAgABAAIAAAAEUjk4AAACAAcAAAAEMDEwMAAAAAAAAgAFAAEAAAABAAAAAAAGAAUAAAABAAAGJgAAAAAAAAAAAAAD6AAHAQMAAwAAAAEABgAAARIAAwAAAAEAAQAAARoABQAAAAEAAAaIARsABQAAAAEAAAaQASgAAwAAAAEAAgAAAgEABAAAAAEAAAaYAgIABAAAAAEAAD6oAAAAAAAAAEgAAAABAAAASAAAAAH/2P/bAIQACAYGBwYFCAcHBwkJCAoMFA0MCwsMGRITDxQdGh8eHRocHCAkLicgIiwjHBwoNyksMDE0NDQfJzk9ODI8LjM0MgEJCQkMCwwYDQ0YMiEcITIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIy/8AAEQgBIAIAAwEiAAIRAQMRAf/EAaIAAAEFAQEBAQEBAAAAAAAAAAABAgMEBQYHCAkKCxAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6AQADAQEBAQEBAQEBAAAAAAAAAQIDBAUGBwgJCgsRAAIBAgQEAwQHBQQEAAECdwABAgMRBAUhMQYSQVEHYXETIjKBCBRCkaGxwQkjM1LwFWJy0QoWJDThJfEXGBkaJicoKSo1Njc4OTpDREVGR0hJSlNUVVZXWFlaY2RlZmdoaWpzdHV2d3h5eoKDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uLj5OXm5+jp6vLz9PX29/j5+v/aAAwDAQACEQMRAD8AjjHNLCPlrrG0y0P/ACxAPtxVVtHth9wFfpXL7BmqxCOdfhG9Mg1LgHaTzVzU9LaCykeKTLcYBFU1zsTPXHNZyi47msJqWw9gNsfGetKV/eZ9BTiu6OM5IwTSnoD9KgoIkzKR6imsBn0qWJWE4ypGR6Uko56UcrBMYFBKe9XNuYxVZR/q8jjNXB908UDISPkH1qaIfJUbD92ccHNSQsAuDWtJ2ZnUV0TBar3K4CEdmqxvWoLlgUXnuK3k9DKK1JV5VfpSx/d/Gmx8qtKhwMe9cZ0j42xIP96ns21gc1CDiUf71OkOJOPxpAR3Dfvs+wpsjnyz6Uy5bEin1FGSYmzQMr78w5x6VXf/AI+STUhfEZFRStiYe4FAx+nHZq0a+rGun6Vy1thdXhP/AE0H611TCtI7GNTcYTTCaeaYaozGcUxgDTyKjNAGbqV09rGCgyScc9qwHuZ7iTBVnJ9K29YH7pPrVSzUeQMAZzQBBFZE4Mp/4CKsqixjCqBVuO1klG77q+pqykEUQ6bm9TTGUo7WSXBAwPU1cjtIouWO9vfpTnnCjriqz3JPCimBbaUKOOBVaS5HQc1WZmbqaTbQArys/Un6UzaakAxRQAwJSYp9IaYEZGO9NxUhppoAux8oPpVy1XKzD/YqpDzGv0q7ZffdfVDTJMmdcSVow8wofaqd0uHNXrbm3T6UIC9pXy6lAf8AaFddtrj7L5LuJvRwf1rtStJiKsyZiNUGStaRco30rPdaQFMr7Um2pytJsqkIiC04LUgSnBaoCMLShalCU8JQBEEpwSpQtOCe1AEQWnbamCU4J7UAQhM04JUwT2p4SmBBspRH7VY2e1KEpAYDR1E0dXCPaoXHtWpzoytVQf2fJ7AH9a58qAK6XU1zp0/+4a51h8ufSuav0OvD7MUL+5XjuRSxkB04pV5hHsaBgMp5rGHxI3lsbqqrIMgHimvbRMOUFTIMxr9Kdtr0rI47sxr23jiKlOOelERZo8lcH61b1GP5EPvV
But when it actually posts to the server (using Square's Retrofit library), I get hundreds of thousands of AAAAAs after this that break the server.
Despite this, though, the logged encoded string doesn't seem long enough to encapsulate all the data that would be associated with a photo from an LG G3.
Use
byte [] data = Base64.decode(base64str);
you would need to reference commons codec from your project in order for that code to work.

Resources