Dynamic image manipulation with UI5 and Promises
Intro
You know that time when manipulating height and width of an image just doesn’t get the job done? This is when a lot of additional players enter the game quickly – HTML5’s canvas, Promises and runtime Image objects.
We’re talking asynchronous content handling here by retaining control over runtime execution along with operating a drawing canvas – all of that in UI5, via JavaScript.
Good thing that we’re going to cover all that ground here 🙂
Although every aspect of client-side image manipulation at runtime would deserve its own blog post (even more so when looked at from a UI5-perspective), let’s go for the sweep through.
Why asynchronous?
Sure thing: working over an image with JavaScript could be done synchronous à la (attention, pseudo-code)
var oImage = Page.getImage();
oImage.resizeTo(1500,1000);
Page.put(oImage);
This would result in the image getting resized – with the user potentially closing the browser window in the process.
Why? Because synchronous JS-operations lock the UI.
If they run for a noticeable amount of time (we’re talking single-digit secons here), it appears to the user that the browser has frozen. This causes frustration, the user might cancel the operation by killing the browser – bad UX.
So the rule of thumb is: when doing extensive JS operations client-side, do them asynchronous in order not to lock up the UI.
Why Promises?
Sure thing: you could use callback functions to go asychronous with your code (attention, pseduo-code again)
var oImage = new Image();
var oImageNew = new Image();
oImage.src = Page.getImage();
oImage.onload = function() {
oImageNew = oImage.resizeTo(1500,1000);
Page.put(oImageNew);
}
Here, the onload event of an image is utilized by an anonymous callback function to resize the image asynchronously (after the original image is ready for processing).
Fine, right? Well, almost.
Because now you certainly want to do other stuff after the resizing has finished.
Which means, you’d have to stuff everything in the anonymous callback function (attention, pseudo-code, you know)
oImage.onload = function() {
oImageNew = oImage.resizeTo(1500,1000);
Page.put(oImageNew);
Page.showPopup("Resizing finished!");
oDownloadButton.show();
// anything else
}
Imagine a couple of asynchronous callback-based operations in a process.
Sooner or later, you end up in callback hell (by now, you should know, pseudo-code and such)
getImage(function(a){
getAnotherImage(a, function(b){
createDownloadLink(b, function(c){
showDownloadButton(c, function(d){
...
});
});
});
});
Wouldn’t it be nice to be able to:
getImage()
.then( resizeIt )
.then( createDownloadLink )
.then( showDownloadButton )
?
Answer: yes, it would be. Awesome even!
And that’s what Promises in JS are for: they let you retain control over sequence in asynchronous operations.
(Technical excursion: compared to callbacks, they give you return and throw back, along with a meaningful runtime stack for debugging. Elaborate on that? Not here. See beginning of blog post, “every aspect … would deserve its own blog post” bla bla).
For the record: Why canvas?
Because literally there is no other way to dynamically manipulate images client-side at runtime.
So, after this lengthy intro:
let’s get started
(Upfront: go grab or look at the source over at GitHub. Way easier to have that open simultaneously in order to get the bigger picture.)
Here’s an image in UI5:
<Image
id="moray_eel"
src="moray_eel.jpg"
press="transformPic" />
We want to do some show-casing manipulation on it (transformPic) after it is clicked.
First step is to get the base64-string representing the image.
That’s where we already dive into the Promise-world (no more pseudo-code from here on, promised (ha ha)):
The Promise
sap.ui.define([
"sap/ui/core/mvc/Controller"
], function (Controller) {
return Controller.extend("vb.controller.main", {
sBase64: '',
transformPic: function (oEvent) {
this.toBase64(oEvent.getSource().getProperty("src"))
// store resulting base64 as property
.then(function (sBase64) {
this.sBase64 = sBase64;
}.bind(this)) // handle context!
// ...
}
})
});
Quick excursion: notice the bind(this) above: in the async world, one key to success is to always watch your context. Meaning, to what scope the this keyword applies to when the code is executed. One way of doing this is to bind the current scope via (duh) bind(this) to a function that is executed at a later point in time, in a different scope.
Back to toBase64(): it is defined as a Promise. Stripped to bare minimum, it looks like:
toBase64: function (sBinaryImgSrc) {
return new Promise(
// do some stuff
// ...
if (everythingOk === true) {
// resolve a.k.a "return" the Promise
resolve(someValue);
} else {
// sth went wrong
// -> reject the Promise a.k.a "report" the error
reject(someError);
}
}
);
}
Promises give you exactly 2 options: resolve or reject them.
When resolving, return a value. Or just return. Important thing is, that you resolve() at all.
When rejecting, think of it as of throw-ing an error.
This is where you regain control: the Promise “spawns” its own thread, executes the stuff you want it to, and then tells you afterwards whether it was successful (resolve) or not (reject). So you get to know, when your async operation has finished – now you can continue doing other stuff (asynchronously) or handle the error:
this.toBase64(oImgSrc)
.then( function(resultOfToBase64) {
// do sth with resultOfToBase64
return doWhatEver(resultOfToBase64);
})
.then( function(resultofWhatEver) {
return doMore(resultofWhatEver); // doesn't have to be a Promise, can be any function returning stuff
})
.then( doEvenMore ) // shorthand notation of calling the Promise doEvenMore with the result of doMore
.catch(function (oError) {
console.log(oError);
});
Following this principle, you can either continue then-ing by
– calling other Promise functions or
– just returning values/functions.
The entire promisey way of then-ing depends on returning values/functions or calling other Promises (which themselves just return stuff).
Read the above again. “The entire…”. And again. “The …”.
Because this is where most errors happen: if your functions/events/Promises are not executed in order, but somehow different, it is most likely you forgot to return/resolve at one point.
(Technical excursion: there’s a whole lot of variations on how to call functions, use more than one parameter for the subsequent then-d call (named functions!) etc pp. As mentioned at the beginning of the post: “every aspect … would deserve its own blog post” bla bla).
Now that we know how to control the sequence of the asynchronous operations, here’s how to manipulate images in UI5 at runtime.
The image manipulation
As mentioned above, using HTML5’s canvas for the purpose is a given. Only so can an image be repainted on a canvas (ha ha).
Here’s where we use the Promises: we load a base64-string onto a canvas, make modifications and then return the modified image, again as base64-string.
Let the source code and the comments speak for themselves:
return new Promise(
function resolver(resolve, reject) {
// construct an "anonymous" image at run-time
var oImage = new Image();
// trigger onload
oImage.src = sBinaryImgSrc;
oImage.onload = function () {
// construct canvas
var oCanvas = document.createElement("canvas");
// make canvas fit the image
oCanvas.width = this.width;
oCanvas.height = this.height;
// reduce opacity of context, then applied to the image
var oContext = oCanvas.getContext("2d");
oContext.globalAlpha = 0.2;
oContext.drawImage(this, 0, 0); // paint the image onto the canvas
// retrieve the manipulated base64-represenation of the image from the canvas
var sBase64 = oCanvas.toDataURL("image/jpeg", 1.0); // retrieve as JPG in 100% quality
// "return" it
resolve(sBase64);
};
// sth went wrong
// -> reject the Promise a.k.a "report" the error
oImage.onerror = function (oError) {
reject(oError);
};
}
)
Following this approach, you can basically manipulate the image any way HTML5’s canvas allows for.
See the example for reducing opacity, inverting color scheme and resizing images (resulting in new images).
Speaking of resizing:
1. the use case for this is simple – say, you allow for picture uploads by your users. For performance reasons, you don’t want to use the original uploaded image everywhere in your UI5 application. Instead, you want a lower-quality version, e.g. for previews. That’s when you need store additional versions of the uploaded image for later use. You can generate as many variations of the uploaded picture at runtime with UI5 and …
2. …the help of using Promises in batch-mode.
What?
Yes, batch-mode. Meaning: trigger multiple asynchronous operations simultaneously and get notified when the last one(!) has finished.
The asynchronous equivalent of the synchronous forEach-loop so to speak.
This is what Promise.all(aPromises) is for. It takes an array of Promises as argument, executes each one and will only resolve() after the last one has finished – returning you the result of every Promise of the argument-array in return. Isn’t that awesome?!?
Look at line 124ff. of the example file for an (duh) example of using Promise.all() – for batch-resizing an image.
Conclusion
So here we are: having used UI5, canvas and Promises in order to manipulate images at runtime.
All of this in an asynchronous way, keeping the UI unlocked, allowing for simultaneous application behavior.
Good stuff!
But especially Promises in JS are hard to understand just by looking at them once. At least for me. It took me some time to get my head around the concept and the coding embedded in UI5. So I encourage you to get your hands dirty as well, fiddle with and/or look at the code and/or look at the screenshots attached.
Happy coding!
tl;dr
use JS Promises in UI5 to load base64-strings on canvas to generate new images.
Hi Buzek,
Why not CSS scale and opacity? - I totally understand the manipulation of colors thou.. As far I'm aware the common practice is to already provide images in their right sizes for different media queries in CSS in order to avoid load / rendering problems / smoothing and anti-aliasing done incorrectly by the browser.
Cheers,
Dan.
PS: you're probably aware but the Canvas down scaling is not super-duper right?
Hi Daniel,
sure thing: if you have access to the image for manual processing, you'd chop it up in different sizes and modify to your liking, then deliver e.g. via CSS sprites. (on a side note: CSS scale does not reduce the size of the image - the entire binary still needs to go across the wire from server -> client)
But the above example use case is that you don't have access to the source images, e.g. when you want to manipulate images on the fly upon user upload. Using canvas certainly is memory-intense - but what other way is there?
Best, V.
Hi Buzek,
On your first point, please notice that your "original" image the bytes in its totality - so there's no impact in there, you may actually manipulate but it does not change the downloaded bytes.. reason for this, you're doing it client side, so if you do a downscale means the client must have had access to the full scaled source in the first place.
On your second point, usually when you have an upload going implies in a server is in the way - which is the often the preferred approach for such since you can use libraries to do this sort of job.. the client could still hold a crop area on the full sized image after upload, but the crop itself happens server side and produces a resource that can later be cached.
The point I'm trying to make is, the scale down "results" in the method suggested provides a bad quality scaled version - you may not have noticed; in fact I believe you didn't - try the same approach with an image that does not have heaps of colors or too much mass and you will see what I'm talking about.. it's not even close to a satisfactory result.
Once again, I understand the point of manipulating the image and the canvas; I'm just making a point about the usage example in this blog..
🙂
Cheers,
Dan.
Hi,
probably there's a misunderstanding here: the use case was never meant to be an example for using variations of an image on a webpage just for display purposes.
Instead, it was intended to provide a "mini Photoshop" use case for manipulating an image. E.g. upload an image, greyscale it, resize it, then download it for further usage.
- V.