Texture paint

    I would like to develop a live texture painting tool. For example,
if you had a terrain mesh, you would be able to use it to paint grass,
dirt, etc,... live in-world.
    I know how to get the face/vertices's the user clicks on, and how
to get the texture coordinates from that. The only problem is the
'live' part. I imagine it wouldn't work by reloading the texture after
every manipulation, since that would be way too slow. Is there any way
I can manipulate an objects texture directly in-memory in
Croquet/Squeak, and make these changes show up in-world?

~JS

On 2/11/08, John Sennesael <[hidden email]> wrote:
You might take a look at the references listed in this note:
https://lists.duke.edu/sympa/arc/croquet-dev/2008-01/msg00007.html

I think that the technique you are looking for involves
changing/remapping the texture coordinates, not rebuilding the texture
to fit the existing coordinates.  This should be fairly easy to do
dynamically.

Dave

It's important to think about what is collaborative (replicated in  
lock-step between participants), and what is not.

You could have everything replicated: the model and it's texture, and  
editing of the texture.
The magic of Croquet is that you don't have to do anything special to  
make the replication part of the programming happen. You just write  
event handlers on the (3d) object to "do the right thing" to self  
material texture. Croquet will automatically make sure that each  
replicated texture has the same thing happen, because what's really  
going is that pointer events are being replicated to every  
participant, and then each participant's computer executes your event  
handler in the same way. (http://opencroquet.org/index.php/ 
The_Core_Model)

This might be a good way to start (and might be the best way to end  
up, if it works for you), but there are several issues that might  
make this fully replicated approach impractical in some applications.

3D vs 2D:
It's hard to write code to paint in 3D. David Faught's answer's link  
(https://lists.duke.edu/sympa/arc/croquet-dev/2008-01/msg00007.html)  
points to some research on how to do this. The conventional  
alternative is to unwrap the texture to 2D. (David's version of  
CCPainter does just that.)

Collaborative vs Local Editing:
An orthogonal issue is whether the editing needs to be collaborative.  
Regardless of whether it's in 3D or 2D, you may find that you get a  
better "feel" by editing the object locally (with lower latency),  
outside the replicated croquet island, and then injecting the result  
back into the island. (Again, that's what CCPainter does.)

Off-island Textures:
Textures in the Croquet SDK can be completely on-island, but this can  
make islands quite large. As a result, making a snapshot and sending  
it to a newly joined participant can take a lot of time. The SDK  
includes a mechanism (two actually) for keeping just a low resolution  
thumbnail on the island, while the renderer is constructed to notice  
this and supply a cacheable high-resolution texture from off-island.  
If you want to make use of that, you need to update the off-island  
texture or media machinery.  Not hard if the editing is local (not  
collaborative), but suppose you were doing collaborative editing,  
where each participant is executing the same code to modify the  
texture.  If the real texture is off-island, what are they editing?  
Furthermore, there needs to be some conceptually central place where  
these off-island textures are kept, so that later participants can  
get them. (A server, a DHT, or whatever.) When we're "done" and want  
to push the result into the off-island texture or media machinery, we  
don't want each replicated bit of code to do upload the same bits to  
that "central" place.  Note that none of this is impossible. It's  
just stuff you have to think about. (http://opencroquet.org/index.php/ 
Off-Island_Resources)

Managing OpenGL:
We don't really want to upload a texture to the graphics card each  
time it is rendered, so it's nice to put it in a buffer and reference  
it as needed. There are some hints to the OpenGL (that I'm not really  
conversant with) to tell OpenGL that the buffer has changed, or that  
it might be changed, etc. One more thing to think about if it becomes  
an issue.
On Feb 11, 2008, at 4:52 PM, John Sennesael wrote:

Technique:
Putting all used textures on one big texture and then dynamically changing the texture coordinates seems the best way indeed. I hadn't thought of that.

3D vs 2D:
The difference here is that I won't be painting 3d objects from 2d images. Instead I will just be changing texture coordinates on a 3d object based on what face the user clicks. As far as I understand it, I don't think any additional texture wrapping/unwrapping will be needed.

Collaborative vs Local editing:
It will probably be best to enter some kind of 'edit' mode where the user can make the changes locally, and then upload the resulting changes. That way I don't have to worry about keeping a 'central' texture for everything. Everyone just edits their local copy and then sends the changes. It would make for easier/faster editing as well.

Off-island textures:
As I understand it, having the texture off-island would make for faster loading time of the island itself? So the user could interact with the world while the textures aren't fully loaded yet? (Instead he'd be seeing a low-resolution 'thumbnail' version?) If that's the case I think it should be something I implement but also something that should be optional. Some content creators might prefer their users to immediately see the end-result and not get a usable world before everything is loaded. This may not have anything to do with the textures being in-island or not though. I guess I could make an optional feature overlaying a loading progress bar, and send some kind of ready message to the island whenever every thing is fully loaded, and still keep the textures in an off-island buffer.

Managing OpenGL:
As stated before, I probably won't be changing the texture itself, but instead I'll be manipulating the texture coordinates. However it would be useful if I can change the base-texture as well. In that case I guess an update flag could be set by a message receiver within the object when a new texture is loaded. The rendering method could just check that update flag and decide whether or not it should reload the texture. I'm not sure if we'd even need those OpenGL features for that, since, using such a flag, we'd only be updating when needed anyway.

~JS


Dave Faught wrote:

I think that the technique you are looking for involves
changing/remapping the texture coordinates, not rebuilding the texture
to fit the existing coordinates.  This should be fairly easy to do
dynamically.

Howard Stearns wrote:

It's important to think about what is collaborative (replicated in lock-step between participants), and what is not.

You could have everything replicated: the model and it's texture, and editing of the texture.
The magic of Croquet is that you don't have to do anything special to make the replication part of the programming happen. You just write event handlers on the (3d) object to "do the right thing" to self material texture. Croquet will automatically make sure that each replicated texture has the same thing happen, because what's really going is that pointer events are being replicated to every participant, and then each participant's computer executes your event handler in the same way. (http://opencroquet.org/index.php/The_Core_Model)

This might be a good way to start (and might be the best way to end up, if it works for you), but there are several issues that might make this fully replicated approach impractical in some applications.

3D vs 2D:
It's hard to write code to paint in 3D. David Faught's answer's link (https://lists.duke.edu/sympa/arc/croquet-dev/2008-01/msg00007.html) points to some research on how to do this. The conventional alternative is to unwrap the texture to 2D. (David's version of CCPainter does just that.)

Collaborative vs Local Editing:
An orthogonal issue is whether the editing needs to be collaborative. Regardless of whether it's in 3D or 2D, you may find that you get a better "feel" by editing the object locally (with lower latency), outside the replicated croquet island, and then injecting the result back into the island. (Again, that's what CCPainter does.)

Off-island Textures:
Textures in the Croquet SDK can be completely on-island, but this can make islands quite large. As a result, making a snapshot and sending it to a newly joined participant can take a lot of time. The SDK includes a mechanism (two actually) for keeping just a low resolution thumbnail on the island, while the renderer is constructed to notice this and supply a cacheable high-resolution texture from off-island.  If you want to make use of that, you need to update the off-island texture or media machinery.  Not hard if the editing is local (not collaborative), but suppose you were doing collaborative editing, where each participant is executing the same code to modify the texture.  If the real texture is off-island, what are they editing? Furthermore, there needs to be some conceptually central place where these off-island textures are kept, so that later participants can get them. (A server, a DHT, or whatever.) When we're "done" and want to push the result into the off-island texture or media machinery, we don't want each replicated bit of code to do upload the same bits to that "central" place.  Note that none of this is impossible. It's just stuff you have to think about. (http://opencroquet.org/index.php/Off-Island_Resources)

Managing OpenGL:
We don't really want to upload a texture to the graphics card each time it is rendered, so it's nice to put it in a buffer and reference it as needed. There are some hints to the OpenGL (that I'm not really conversant with) to tell OpenGL that the buffer has changed, or that it might be changed, etc. One more thing to think about if it becomes an issue.
On Feb 11, 2008, at 4:52 PM, John Sennesael wrote:

I just realized there's 1 problem with changing just the texture
coordinates... You can't get very detailed using this approach...
You'd only be able to paint entire faces with the texture....