Roku Developer Program

Join our online forum to talk to Roku developers and fellow channel creators. Ask questions, share tips with the community, and find helpful resources.
cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
coolbits
Visitor

Roku 2D API; Capture what is on screen

Hi,

I want to capture a section of the image currently displayed on the screen. Let's say I have a sprite and called the screen.Finish() function. Now that image is drawn in the screen. What I want to do is, capture a section of that image (from the screen) and draw it again and again.

Using the 2D api ( or any other mechanism), how may I achieve it?

Thank you,
Chandana
0 Kudos
13 REPLIES 13
NewManLiving
Visitor

Re: Roku 2D API; Capture what is on screen

First of all if you are using a double buffered
roScreen, You only need to call SwapBuffers. You do not have to call
Finish since SwapBuffers calls Finish for you. Second if you are doing
Any type of animation which requires rapid movement you must use
A double buffered screen. If you do not you will tear the bitmap. Third,
You only need one instance of a Compositor and not one for each bitmap
Compositors are for creating sprites. So you would create all your bitmaps
Then create regions for those bitmaps and finally create the sprites you
Need using the one compositor. Call the compositors drawall function
To paint all those sprites you just added. Then call swapbuffers to display
It to the screen. Hopefully you got that because your original code samples
Indicate otherwise
Now to replicate an area of a bitmap is easy
1. Determine the coordinates for the area that you want to replicate
2. Create a new region for that area
3. Use the Single instance of rocompositor to create a sprite for that
Region. Place it At the location on the screen that you want it
3. Call the compositors drawall function
4. Call swapbuffers
5. To change locations on the screen call the sprites moveto function
6. Repeat steps 3 and 4
6 when you are done with the sprite you created you must call
The sprites remove function to remove it from the compositor. If you
Do not and you are frequently recreating these sprites then You will
Run out of memory
My Channels: 2D API Framework Presentation: https://owner.roku.com/add/2M9LCVC
Updated: 11-11-2015 - Completed Keyboard interface
The Joel Channel ( Final Beta )
0 Kudos
RokuMarkn
Visitor

Re: Roku 2D API; Capture what is on screen

Hm, seems kind of unnecessarily complicated to use compositor/sprites for this. I would stop at step 2 and then just call DrawObject to copy the region to another bitmap.

--Mark
0 Kudos
NewManLiving
Visitor

Re: Roku 2D API; Capture what is on screen

I believe that he is already using a compositor Mark
If you look at his original code Also he mentions
A sprite in this post. You cannot create a sprite
Without a compositor That said you really
Are adding complexity if you started with a compositor
And then begin to draw bitmaps directly onto the screen
In your solution what would you do with the bitmap
Since the compositor covers the entire screen
If it's setdraw is to the screen. Now if he is just
Using the term sprite to mean bitmap
And he is not already using a compositor
Then your point is well taken
My Channels: 2D API Framework Presentation: https://owner.roku.com/add/2M9LCVC
Updated: 11-11-2015 - Completed Keyboard interface
The Joel Channel ( Final Beta )
0 Kudos
TheEndless
Channel Surfer

Re: Roku 2D API; Capture what is on screen

"NewManLiving" wrote:
I believe that he is already using a compositor Mark
If you look at his original code Also he mentions
A sprite in this post. You cannot create a sprite
Without a compositor That said you really
Are adding complexity if you started with a compositor
And then begin to draw bitmaps directly onto the screen
In your solution what would you do with the bitmap
Since the compositor covers the entire screen
If it's setdraw is to the screen. Now if he is just
Using the term sprite to mean bitmap
And he is not already using a compositor
Then your point is well taken

The compositor won't obscure everything on the screen, unless you have a full screen sprite in it. You can also draw to the screen after you draw with the compositor, so if the new bitmap is meant to have a higher z-order, you could draw it directly. If I'm honest, I find using the compositor, in general, to be a bit overly complex for many, if not most, 2D implementations.
My Channels: http://roku.permanence.com - Twitter: @TheEndlessDev
Instant Watch Browser (NetflixIWB), Aquarium Screensaver (AQUARIUM), Clever Clocks Screensaver (CLEVERCLOCKS), iTunes Podcasts (ITPC), My Channels (MYCHANNELS)
0 Kudos
coolbits
Visitor

Re: Roku 2D API; Capture what is on screen

Hi All,
Thank you for your replies. But I don't think my problem is still clear. Sorry for not explaining the problem well. I will re explain it here by an example.

Let's say I have a compositor and two sprites in it. Say I position two sprites such a way that, they overlap. Now, in the display(TV screen), I have a full view of one sprite and section of the other sprite(because they are overlapped).

Now, let's say, I want to capture a region of what is being displayed on the TV screen. For an example, I want to capture a rectangle where both sprites are met. Further explaining, that rectangle will have a section from one sprite and a section from the other.

I believe now my problem is somewhat clear.

Thank you,
Chandana
0 Kudos
NewManLiving
Visitor

Re: Roku 2D API; Capture what is on screen

"TheEndless" wrote:
"NewManLiving" wrote:
I believe that he is already using a compositor Mark
If you look at his original code Also he mentions
A sprite in this post. You cannot create a sprite
Without a compositor That said you really
Are adding complexity if you started with a compositor
And then begin to draw bitmaps directly onto the screen
In your solution what would you do with the bitmap
Since the compositor covers the entire screen
If it's setdraw is to the screen. Now if he is just
Using the term sprite to mean bitmap
And he is not already using a compositor
Then your point is well taken

The compositor won't obscure everything on the screen, unless you have a full screen sprite in it. You can also draw to the screen after you draw with the compositor, so if the new bitmap is meant to have a higher z-order, you could draw it directly. If I'm honest, I find using the compositor, in general, to be a bit overly complex for many, if not most, 2D implementations.


A compositor that is setdraw to the screen initially clears the screen with the background color. It knows nothing of any bitmaps that you place on the screen. If you should create them later you would have to manage the compositor as well as the bitmaps, so it defeats the purpose of the compositor. I have found in past experience (I have tried many things before I came to the conclusion that the compositor is the best way ), that if you begin to create bitmaps directly onto the screen (over the compositor) and you start to accumulate them, then you may run into a further problem with the compositor possibly clearing the screen instead of only the background of the sprite. This may be due
to the algorithm used to determine the intersection of regions or rectangles. I know when I programmed video memory in C++ if the intersection of regions or rectangles exceeded a certain percentage of the display then I would just clear the entire screen. This was common practice. I cant say this is the case, but I did have problems.
As far as the compositor being too complex , I must disagree. It is a container class that is elegant and optimized for speed. It draws all its sprites internally so it is fast and clean. It is by far the best way to implement a stack of windows (just create a compositor on top of another one). It has built in animation and collision testing. If you could create a bitmap and a region, you can certainly create a compositor. It may look like more code, but you only have two calls to draw everything. As far as your statement concerning "many if not most 2d implementations" i'm not quite clear what you mean by this: the compositor is part of the 2d API for a very good reason. Unfortunately, I get the impression that many people cannot figure out why. Lastly, our friend here was already using the compositor as I stated earlier, so naturally I answered his question in light of what he was already using
My Channels: 2D API Framework Presentation: https://owner.roku.com/add/2M9LCVC
Updated: 11-11-2015 - Completed Keyboard interface
The Joel Channel ( Final Beta )
0 Kudos
TheEndless
Channel Surfer

Re: Roku 2D API; Capture what is on screen

"NewManLiving" wrote:
A compositor that is setdraw to the screen initially clears the screen with the background color.

Try a transparent background color. I haven't tried it recently, but I was originally drawing the Aquarium Screensaver background to the screen, before using the compositor on top of it, but I later changed the background bitmap to be a sprite. The behavior may have changed since then.
"NewManLiving" wrote:
if you begin to create bitmaps directly onto the screen (over the compositor) and you start to accumulate them, then you may run into a further problem with the compositor possibly clearing the screen instead of only the background of the sprite. This may be due
to the algorithm used to determine the intersection of regions or rectangles. I know when I programmed video memory in C++ if the intersection of regions or rectangles exceeded a certain percentage of the display then I would just clear the entire screen. This was common practice. I cant say this is the case, but I did have problems.

I'm not really sure what you mean by all of that, but my point was really only that you don't have to use the compositor exclusively. It's not all or nothing.
"NewManLiving" wrote:
As far as the compositor being too complex , I must disagree.

I find that it makes managing the memory a lot more tedious. I also found that calling Draw() instead of DrawAll() rarely works as expected, which defeats one of the major intended benefits (i.e., only having to draw regions of the screen that have changed). Draw() is also slower than DrawAll(), which baffles me, and must be a bug.
"NewManLiving" wrote:
As far as your statement concerning "many if not most 2d implementations" i'm not quite clear what you mean by this: the compositor is part of the 2d API for a very good reason.

This probably boils down to how I've implemented my base framework, which also allows me to draw the whole screen with a single line of code. Unless you're working with strictly bitmaps, and a fair number of them, in most cases, the compositor doesn't add a lot of benefit (for me). If the compositor supported text sprites, simple shape sprites, and scaling, I might be inclined to use it more often, but as it stands now, the only way to accomplish any of those is to create a completely separate bitmap in-memory before making it a sprite, and drawing text to an in-memory bitmap not only requires even more graphics memory, it also results in less than stellar results (bad anti-aliasing). Likewise, if I want to fade out a portion of the screen, the only way to do that is to create a bitmap the size I want to fade, and add it as a sprite to the compositor, instead of simply calling DrawRect() on the screen itself. If I want to partially fade the whole screen to emphasize a single sprite (like the fish info display in the Aquarium Screensaver), that's a whole lot of unnecessary graphics memory usage.
"NewManLiving" wrote:
Unfortunately, I get the impression that many people cannot figure out why.

Honestly, I think it was primarily added for game development, which I don't do. That being said, I have used it on a number of occasions. I just find it easier to draw directly to the screen in most cases.
"NewManLiving" wrote:
Lastly, our friend here was already using the compositor as I stated earlier, so naturally I answered his question in light of what he was already using

Which is a fair point, but he/she never mentioned using the compositor in this thread, so I'm not sure where you got that information. He/she mentioned a sprite, but that's a pretty standard term for an onscreen graphical element. Even still, it still goes back to my main point that, just because you're using the compositor doesn't mean that you have to use it for everything on screen.

Either way, I don't think you can actually grab a region from roScreen for drawing elsewhere on the screen or a bitmap. I haven't tried it recently, but the last time I tried that, it just returned a blank region. I'd be curious to know if it works now (I guess I could try it, but I'm far too lazy for that).

And finally, with all of that being said, I've run into some performance issues recently on the slower platforms that I'm hopeful may be resolved (or at least mitigated) by using the compositor, so I've been considering re-visiting it. When I get around to doing that, my tune may vary well change... 😉
My Channels: http://roku.permanence.com - Twitter: @TheEndlessDev
Instant Watch Browser (NetflixIWB), Aquarium Screensaver (AQUARIUM), Clever Clocks Screensaver (CLEVERCLOCKS), iTunes Podcasts (ITPC), My Channels (MYCHANNELS)
0 Kudos
NewManLiving
Visitor

Re: Roku 2D API; Capture what is on screen

Coolbits was always asking questions about the API and had already posted a number of code samples using the compositor. Even just recently. So naturally the "sprite" mentioned here more than likely referred to a compositor sprite, Again, as I stated in my response to Mark, if he meant bitmap then Mark's point was valid. As far as the Compositor's use for games, the whole API, I believe was intended for games, as I also mentioned to Coolbits in past posts. If you ever developed components in other languages the term composite is a very common one. So it is a natural fit for me. The compositor also lends itself perfectly for developing modular applications. You can either create a new roScreen ( worse, if even possible, never tried it ) , or you can clear off one instance of roScreen for another part of your application, and then have to redraw everything when the user wants to return to the previous screen (bad),or you can create modular objects which contain an instance of their own compositor and simply layer them on top of each other, or if memory is an issue you can swap them out as I do. (much better). Any memory issue I have come across (and my channel takes the API to the limit) can be addressed with the modular approach and the ability to remove sprites from the compositor. The compositor has built in layering abilities which enables you to place your sprite anywhere at anytime without having to address anything else above or below it. Much like having the imagecanvas on fire. You also mentioned that most (or many) or (if not all) applications do not need to use it. That is a broad statement. The point of using the "game" API is to develop a custom channel (for most of us on this board) this means that even a trivial one requires some amount of complexity. To create a modular channel, such as I am doing , I really would have to develop a framework component to manage all my drawing, probably similar to what you eluded to in your framework. Maybe I do things differently. I like the freedom of the API so I combine the best of both worlds: application development and animation. My examples are just a small sample of what I have developed that can be duplicated with just a bitmap and some regions . However, in my own development I have much more complex objects, For example, one grid that I employ uses an animated selector that is a complexity in itself, since it is also a one row/col grid with multiple regions to create a type of masked effect for my selector. My grids can also create dynamic rows which are also animated. My listview menus have the ability to slide anywhere at any layer. With this type of complexity, I am freed form having to remember where everything is and what needs to be redrawn, simply by using the compositor. And lastly, as far as what coolbits said, nobody understood is initial request, and I do not believe that you can capture any part of video. I memory serves me correctly, you cannot even do this with the screensaver utility. This may have changed I don't know
My Channels: 2D API Framework Presentation: https://owner.roku.com/add/2M9LCVC
Updated: 11-11-2015 - Completed Keyboard interface
The Joel Channel ( Final Beta )
0 Kudos
TheEndless
Channel Surfer

Re: Roku 2D API; Capture what is on screen

"NewManLiving" wrote:
The compositor also lends itself perfectly for developing modular applications.

All of my code is modular, as well, which would actually make it fairly easy for me to add the compositor as an option, so I think I might give that a go this weekend. You're passionate enough about it, that you've almost convinced me... 😉 I'm also a bit biased/jaded at the moment, since the last few 2D implementations I've done have included video playing at the same time, which takes the bitmap memory management to a whole new frustrating level.

We should probably take this to another thread, but I'm curious as to how you do text, simple rects, animated fades, and animated scaling up (not down, as that wouldn't technically require a change in bitmap size) with the compositor. And how can you do any of that without using significantly more memory? If you're as interested in discussing as I am, let's start a new thread.
My Channels: http://roku.permanence.com - Twitter: @TheEndlessDev
Instant Watch Browser (NetflixIWB), Aquarium Screensaver (AQUARIUM), Clever Clocks Screensaver (CLEVERCLOCKS), iTunes Podcasts (ITPC), My Channels (MYCHANNELS)
0 Kudos
Need Assistance?
Welcome to the Roku Community! Feel free to search our Community for answers or post your question to get help.

Become a Roku Streaming Expert!

Share your expertise, help fellow streamers, and unlock exclusive rewards as part of the Roku Community. Learn more.