The Challenges of writing an OpenSim client in Unity3D
Rob Smart | November 12, 2009I thought I’d write a quick article on the possibilities of writing a client for OpenSim using Unity3d, I’m writing this as there has been a lot of discussion around this recently after the Unity3d indie version became free.
I’ve done a fair bit of prototyping and investigation on this in the past, so I’ll share what I’ve discovered and where I have got to.
Firstly I want to make clear a couple of points about the way Unity3d applications can be deployed. One option for creating a client is standalone mode, in this the produced client will be an executable desktop app that works on Windows or OS X. The second and most attractive option for an OpenSim client is deploying as a Web Plugin that can be embedded in any webpage.
Now I’ve made these two options clear I’ll point out the additional difficulties of creating a Web Plugin client. Firstly any Unity3d app deployed as a web plugin is security sandboxed just as a java applet would be. In practical terms what this means is that certain standard API calls in the embedded Mono engine are not allowed. For example one fairly critical class that can’t be used is System.Net.WebRequest. As far as I can tell there is not a list anywhere of API functions that do not work, it’s a case of trial and error and trawling the Unity forums.
The second issue with the web plugin is that you can’t use any additonal C/C++ libraries you can however use c# dlls. The restriction this causes though is that your included c# dlls can’t rely on any platform specific external libs, think System.Drawing and libgdi.
So now lets look at what we need to implement an OpenSim client. First off we’re going to need to communicate with the server so we need libOMV, for creating a standalone unity application using a pre 0.7 libomv libary there is no problem, it works straight out of the box. Why pre 0.7.0 ? Because in 0.7.0 some features of .NET 3.5 have been introduced (see this opensim-dev post for detail) and Unity3d is on a back level version of mono, i’m not sure exactly which but i think it’s around 2.0.
It is possible to work around these changes and get libomv 0.70 working in Unity for a standalone client. However try deploying your simple log on and chat client as a Web Plugin and you will soon find that you don’t even get to the login stage as libOMV uses HttpWebRequest which doesnt work in the browser plugin. The fix for this is to rewrite parts of libOMV to use the unity WWW object instead.
So next up we’ve got the issue of 3D object formats which luckily is solved (in the most part) by the availability of the PrimMesher library written by Dahlia. I spent some time the other week getting this working and shared the Unity side of things with Dahlia who has gone on to get sculpties working as well. The remaining issue with objects is texturing and more specifically the decoding of jpeg2000 images. At the moment I’m decoding these to regular jpegs on the server side before assigning the textures in Unity.
I guess that leads onto the discussion of whether to bother with trying to get libOMV working in the browser, personally i think it’s worth it. Many would disagree as they don’t see a client embedded in the browser as any different from a desktop app, however this argument is neutralized somewhat by the fact that Unity can communicate via javascript with the page it is embedded into, this leaves open the possiblity of widgetized system for things like inventory management, chat etc. etc. and custom widgets for different opensim grids.
The alternative option of course is to write a Unity specific client library and implement the matching iClientApi, this may in fact be a better solution as keeping a forked libOMV version just for unity will be a bit of a pain. The down side of this alternative is that the client won’t work with the main Second Life grid.
So that’s the end of this little brain dump,I hope it has informed anyone thinking of tackling a Unity client for OpenSim and also hope it hasn’t put you off at all.
Hi Rob – interesting post, obviously there are a lot of folk keen on a browser-based interface for OpenSim. It’s not clear to me what advantage there would be in the Unity3D route compared to the 3Di Rei code that is out there – was this just a technical ‘can this be done’ or is there something about Unity3D that would make it a preferable route to go down?
OK, my memory of Unity wasn’t good enough. Clearly being able to get the power of that graphics engine would be a ‘very good thing’ compared to what I saw last time I tried Rei…
Good question Neil.
Personally I haven’t dug around in the 3Di Rei code yet but it looks promising from what i have read/seen on blogs etc. The benefits of the 3Di viewer are that everything in it is open and of course free this leaves more flexibility for changes down at the graphics engine level.
With Unity3d you have to rely on the provided APIs which are vast and flexible though at some point may prevent you from doing something. However with Unity you’ll be piggy backing on the extensive testing done by the Unity3d guys on hundreds of graphic cards which is bound to reap rewards in terms of stability of the client. From my experience it is a lot quicker to write games/clients with Unity3ds c# and javascript API support than it is doing pretty low level c/c++ development with the ogre3d or irrlicht engines. You can still do pretty low level stuff with Unity (creating shaders or doing direct GL commands)
There are pros and cons to all of these virtual world client solutions, writing a client is never going to be an easy business and i guess that’s why i favour unity as its APIs and editor environment make some of the heavy lifting a lot easier.
Hi Rob,
This is a great summary of the things you found with Unity3D. It’s interesting that WebClient doesn’t work and i wonder if it might be possible to circumwent that sandbox by changing the way the Unity webapp is registered to the browser (it is possible to indicate a secure context for ActiveX – although it will still refuse to elevate from Low privileges, so you need to be careful about File I/O – but I’m not really sure about other technologies). Also, I would miss the ability to use native libraries (if I understand correctly, P/Invoke is not possible?). For Jpeg2000, CSJ2K might be a good solution on the client side if you can’t use native code with wrappers.
Neil:
I guess what Rei really lacks right now is some kicking demo contents – but that will change soon.
Unity3D has a powerful graphics engine and great demo contents. Rei has a similarly powerful graphics engine, and the main issue lies in the contents. I can point to amazing games running on Irrlicht (or Ogre), but as long as OpenSim doesn’t support/have the sort of content (high quality meshes with custom shaders, environment-maps, lighting, etc), that is really irrelevant. Of course with Rei, you can always write custom SceneNodes or shaders, even custom logic with Plugins and there is already a similar experience to unity’s API with Javascript integration. And if you are still unsatisfied, you can always pick the engine apart – although having done that a couple of times, I’m not sure I’d recommend it
Well, I guess – as Rob said – in the end what it comes down to is weighing the pros and cons between super-easy and super-flexible.
I ran into the same problem with Unity and LibOMV last week : the last Unity 2.6 uses a too old Mono version (2.0).
An Unity project can be integrated in a web browser but also it can be build in a stand-alone viewer (an exe file) like SL, Hyppo or Naali. And it can even be deployed on an iPhone.
Ogre and Irrlicht are 3D engines. Unity is a game engine; it mean that it also includes Physics, Network, AI, Animation, GUI, Terrans/Levels Editor, Audio, etc …
3Di’s Rei viewer is built on Idealist viewer available on the opensimulator forge, http://forge.opensimulator.org/gf/project/idealistviewer/ and while the viewer itself might be opensource, the server that it connects to to use all the cool functionality is not. while yes it does connect to OpenSimulator, quite a bit of its functionality is reduced if its not connected to the 3Di Enterprise Version of OpenSimulator, which i do not believe is open source at this time. If your interested in fiddling with the core of Rei viewer i suggest you check out idealist project 1st, it doesn’t require any closed source software for 100% functionality. and just to clarify Irrlicht and Ogre are not the same thing at all as it was suggested somewhere above.
[...] Naked Imagination · The Challenges of writing an OpenSim client in Unity3D (tags: opensim Unity3d) [...]
[...] flexibility both sides of the equation that makes them very solvable. This is a high level view, Rob has some more detail here on the challenges. Add in some interoperation definition with Vastpark to help bind the two and make some mappings. [...]
Hi
I’m developping 3d worlds with Unity 3D and I have also a grid with opensim…i’m looking for a solution to open opensim with unity3D …
Can you tell me if you’re viewer is usefull at this moment ??
Thank you
Hi Rob,
very interesing article! I am also looking for a uinity3d-viewer working with opensim. Is it possible to download the viewer somewhere?
Thanks in advance!
Interesting section on VOIP at home. I have a Cisco ATA box that was supplied by Vonage but it has a password on it that I can not seem to uncover. I want to use the Cisco bax as my VOIP gateway but need to access it.
Any ideas?
Thanks
Alastair
Hi Alastair, I’m not familiar with that hardware. I guess it’s possible you might be able to do a factory reset and then use whatever the hardwares default credentials are, a bit of Googling might turn something up.