Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] env.render() giving no/wrong results #2889

Closed
1 task done
wookayin opened this issue Jun 14, 2022 · 3 comments
Closed
1 task done

[Bug] env.render() giving no/wrong results #2889

wookayin opened this issue Jun 14, 2022 · 3 comments

Comments

@wookayin
Copy link
Contributor

wookayin commented Jun 14, 2022

Describe the bug

After the merge of new render API #2671 which is not stable yet, env.render() giving no results when it is called more than once. Also the rendering result is not correct in terms of type and shape. Please see the example below.

Code example

env = gym.make("HalfCheetah-v4", render_mode='rgb_array')
env.reset()
im = env.render(width=200, height=200)
im2 = env.render()

I see three problems here:

  • im contains a python list of [480x480x3] ndarray: im[0].shape == (480, 480, 3). In our early version of gym, im itself was a ndarray. I don't think that rendered images contained in a list is correct; and the behavior is different.

  • The width/height parameter env.render(width=200, height=200) are ignored; the rendering dimensions are wong.

  • The next call of env.render() will give no results: it returns an empty list, i.e. im2 == []

System Info

  • gym: latest HEAD 053ee80
  • installed from source
  • macOS
  • python 3.10

Additional context

/cc @younik who is the author of #2671.

  • I have checked that there is no similar issue in the repo (required)
@wookayin wookayin mentioned this issue Jun 14, 2022
1 task
@wookayin wookayin changed the title [Bug] env.render() giving no/wrong results after the first invocation [Bug] env.render() giving no/wrong results Jun 14, 2022
@younik
Copy link
Contributor

younik commented Jun 14, 2022

Hello,

im contains a python list of [480x480x3] ndarray: im[0].shape == (480, 480, 3). In our early version of gym, im itself was a ndarray. I don't think that rendered images contained in a list is correct; and the behavior is different.

This is done on purpose; with the new render API, the method render returns, in general, a collection of frames. You can do some steps without calling render to see this. Instead, if you want to collect a single frame, you should use single_rgb_array.

The width/height parameter env.render(width=200, height=200) are ignored; the rendering dimensions are wong.

When you use the new render API, those arguments should be passed to __init__. Tho, you are right, they are still not implemented in __init__; I will push a fix soon, thanks for spotting it.
The legacy code still works with dimensions (don't specify render_mode to use it).

The next call of env.render() will give no results: it returns an empty list, i.e. im2 == []

This is the expected behaviour since the render call pops the frames from the collection. This is done to allow users to clean the frame collection.

@wookayin
Copy link
Contributor Author

wookayin commented Jun 14, 2022

My apologies I wasn't following the history #2540 #2671 carefully enough:

In case render_mode != None, the argument mode of .render() is ignored.
In case render_mode = "human", the rendering is handled by the environment without needing to call .render().
With other render modes, .render() returns a proper List with all the renders since the last .reset()or .render(). For example, with render_mode = "rgb_array", .render() returns a List of np.ndarray, while with render_mode = "ansi" a List[str].

Maybe it's too late to argue against the new API, but I don't feel this new API makes sense. To normal users it is an unexpected and surprising breaking change made on minor releases, and fool-proof guards are lacking. This might be just a minority opinion of mine, but I would like to strongly argue that we should allow rendering arguments as in the previous behaviors.

This is done on purpose; with the new render API, the method render returns, in general, a collection of frames.

Why? This sounds strange. In which scenario/use cases this would be useful? Is it something really necessary at the cost of breaking backward compatibility and all the existing codes? Is it because one would want to render all the frames when there is frame-skip? If it's needed for some reason, why didn't we have it under a different render_mode name (say multiple_rgb_array)?

I can imagine how rendering is done internally, but if you would like to get the frames in the case of frame skips, those should have been available under different APIs (or different names) or use cases, such as using a different render mode other than rgb_array. This is really, really confusing (nor documented well). I think env.render("rgb_array") returning the current scene renderred is the correct and backward-compatible behavior, which should be able to be called multiple times as well.

@wookayin
Copy link
Contributor Author

Since this is not a bug as per #2671, I will close this issue. Let me move and continue my argument to #2671 to centralize the discussion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants