robsmart.co.uk

I thought I’d write a quick article on the possibilities of writing a client for OpenSim using Unity3d, I’m writing this as there has been a lot of discussion around this recently after the Unity3d indie version became free.

I’ve done a fair bit of prototyping and investigation on this in the past, so I’ll share what I’ve discovered and where I have got to.

Firstly I want to make clear a couple of points about the way Unity3d applications can be deployed. One option for creating a client is standalone mode, in this the produced client will be an executable desktop app that works on Windows or OS X. The second and most attractive option for an OpenSim client is deploying as a Web Plugin that can be embedded in any webpage.

Now I’ve made these two options clear I’ll point out the additional difficulties of creating a Web Plugin client. Firstly any Unity3d app deployed as a web plugin is security sandboxed just as a java applet would be. In practical terms what this means is that certain standard API calls in the embedded Mono engine are not allowed. For example one fairly critical class that can’t be used is System.Net.WebRequest. As far as I can tell there is not a list anywhere of API functions that do not work, it’s a case of trial and error and trawling the Unity forums.

The second issue with the web plugin is that you can’t use any additonal C/C++ libraries you can however use c# dlls. The restriction this causes though is that your included c# dlls can’t rely on any platform specific external libs, think System.Drawing and libgdi.

So now lets look at what we need to implement an OpenSim client. First off we’re going to need to communicate with the server so we need libOMV, for creating a standalone unity application using a pre 0.7 libomv libary there is no problem, it works straight out of the box. Why pre 0.7.0 ? Because in 0.7.0 some features of .NET 3.5 have been introduced (see this opensim-dev post for detail) and Unity3d is on a back level version of mono, i’m not sure exactly which but i think it’s around 2.0.

It is possible to work around these changes and get libomv 0.70 working in Unity for a standalone client. However try deploying your simple log on and chat client as a Web Plugin and you will soon find that you don’t even get to the login stage as libOMV uses HttpWebRequest which doesnt work in the browser plugin. The fix for this is to rewrite parts of libOMV to use the unity WWW object instead.

So next up we’ve got the issue of 3D object formats which luckily is solved (in the most part) by the availability of the PrimMesher library written by Dahlia. I spent some time the other week getting this working and shared the Unity side of things with Dahlia who has gone on to get sculpties working as well. The remaining issue with objects is texturing and more specifically the decoding of jpeg2000 images. At the moment I’m decoding these to regular jpegs on the server side before assigning the textures in Unity.

I guess that leads onto the discussion of whether to bother with trying to get libOMV working in the browser, personally i think it’s worth it. Many would disagree as they don’t see a client embedded in the browser as any different from a desktop app, however this argument is neutralized somewhat by the fact that Unity can communicate via javascript with the page it is embedded into, this leaves open the possiblity of widgetized system for things like inventory management, chat etc. etc. and custom widgets for different opensim grids.

The alternative option of course is to write a Unity specific client library and implement the matching iClientApi, this may in fact be a better solution as keeping a forked libOMV version just for unity will be a bit of a pain. The down side of this alternative is that the client won’t work with the main Second Life grid.

So that’s the end of this little brain dump,I hope it has informed anyone thinking of tackling a Unity client for OpenSim and also hope it hasn’t put you off at all.

A couple of months back i spent some time writing a module for OpenSim, the purpose of this module being to allow you to connect directly to an MQTT messaging server from within the OpenSim scripting language. Thereby enabling real-time Publish Subscribe messaging in a 3d environment.

Adding messaging capability to the scripting language opens up a raft of possibilities for integrating other live systems with in world objects, for previous demonstrations I have implemented chat between virtual worlds and also
synchronous presentations across words and opensim regions amongst other things.

For this one I heard of some live data available from one of our local master inventors Andy Stanford-Clark on the positions of Ferries around the Southampton, Portsmouth and the Isle of Wight. He was publishing these message onto his MQTT microbroker which meant with my module for OpenSim i could show the positions of all the local shipping in near real-time along with the name of the ship and other information like course speed and cargo type.

In order to show this data i first terraformed an island within the IBM opensim grid to look like the Isle of Wight and surrounding sea. I did this by creating an image in photoshop and then using OpenSims terrain import feature. The image was based on a satellite photo of the area.

This is the resulting opensim island

The next step was to write a script for the ships, this script took the description of the object(ferry) it was assigned to and used it to subscribe to the information for that ship. Each description is the pubsub topic for the ship e.g. ferries/Isle Of Wight/Red Cat 1

The script listens for incoming messages then moves the ship object to the correct location on the simulated region by converting latitude and longnitude into an X Y position. It also takes the bearing and makes the ship face in the correct direction.

here’s a couple more shots of the ferries in action

The Ferries currently live on the IBM internal opensim grid (sorry only accessible to folks inside the IBM firewall )

In OpenSim you’re not just restricted to using LSL for scripting, it’s also possible to use c#. This opens up the possiblilities for far more powerful scripts that can access c# in built libraries.

For example I’ve previously mentioned that the string processing methods in Second Life are not very flexible when it comes to reading formatted data. Thus the reason I implemented the osParseJSON method to make Web APIs easier to use.

Obviously another common data format is XML, in this case I don’t have to implement any new OpenSim script functions because the XML capabilities are available natively in c#.

Here is a simple example of reading the RSS feed for my blog, and reading out the entries in chat.


//c#
// displays the contents of an RSS 2.0 feed

string URL = “http://robsmart.co.uk/feed”;
System.Xml.XmlDocument xDoc = new System.Xml.XmlDocument();
LSL_Types.LSLString requestID;

public void default_event_state_entry()
{
llSay(0,”RSS reader current Feed is ” + URL);
getFeedContent();

}

public void getFeedContent()
{
requestID = llHTTPRequest(URL, new LSL_Types.list(), “” );
}

public void default_event_touch_start(LSL_Types.LSLInteger total_number)
{
// read out the RSS feed.
displayFeed();

}

public void displayFeed()
{
System.Xml.XmlNodeList items = xDoc.GetElementsByTagName(“item”);

for(int i=0;i < items.Count ; i++)
{
string title = items[i].SelectSingleNode(“title”).InnerXml;
string link = items[i].SelectSingleNode(“link”).InnerXml;

string description=”no description available”;

if(items[i].SelectSingleNode(“description”)!=null)
description = items[i].SelectSingleNode(“description”).InnerXml;

llSay(0,title + “\n” + description + “\n”);
}
}

public void default_event_http_response(LSL_Types.LSLString request_id, LSL_Types.LSLInteger status, LSL_Types.list metadata, LSL_Types.LSLString body)
{
if (requestID == request_id)
{
// store the xml
xDoc.LoadXml(body);

// process the xml
llOwnerSay(“loaded feed”);
}
}

Tish Shute very kindly asked me to do an interview for her excellent blog over on Ugotrade. In the interview I discuss where OpenSim fits in with the rest of the Web and particularly how some of the recent work i’ve been doing makes it a more viable platform for consuming data and services from websites.

You can find the interview here.

As described over on Eightbar I have made my first contribution to the OpenSim opensource project. The contribution is in the form of a new scripting function called osParseJSON. This function allows a c# script in OpenSim to consume the JSON notation provided by many of the major Web 2.0 APIs provided by services such as Flickr and Google translation.

The following example is a script that uses the Google Translate API to let an OpenSim avatar translate ther conversations between 23 different languages.

(disclaimer – please read the terms of conditions of the Google translate API and abide by them)

//c#
// This script is written as an example use of the osParseJSON method
// it uses the Google translate API
// ensure you have read the terms and conditions of the Google translate API
// http://code.google.com/apis/ajaxlanguage/documentation/

LSL_Types.key requestID;
string sourceLang = "en";
string targetLang = "fr";

public void default_event_state_entry()
{
     llSay(0,"translator running say '/1 sentence' to translate something");
     llSay(0,"translator running say '/2 source langage' to change target language e.g. '/2 fr'");
     llSay(0,"translator running say '/3 target langage' to change source language e.g. '/3 en'");
     llSay(0,"translator running say '/4 help', to list languages");
     llListen(1, "", NULL_KEY, "");
     llListen(2, "", NULL_KEY, "");
     llListen(3, "", NULL_KEY, "");
     llListen(4, "", NULL_KEY, "");
}

public void default_event_touch_start(LSL_Types.LSLInteger total_number)
{
     llSay(0,"translator running say '/1 sentence' to translate something");
     llSay(0,"translator running say '/2 source langage' to change target language e.g. '/2 fr'");
     llSay(0,"translator running say '/3 target langage' to change source language e.g. '/3 en'");
     llSay(0,"translator running say '/4 help', to list languages");
}

public void default_event_http_response(LSL_Types.LSLString request_id, LSL_Types.LSLInteger status, LSL_Types.list metadata, LSL_Types.LSLString body)
{
        if (requestID == request_id)
        {
            // the Google JSON string returned wil be of the format
            //  {"responseData": {"translatedText":"Bonjour"}, "responseDetails": null, "responseStatus": 200}
            // call the osParseJSON method so we can read the contents 
            System.Collections.Hashtable response = (System.Collections.Hashtable) osParseJSON(body);
            System.Collections.Hashtable responsedata = (System.Collections.Hashtable) response["responseData"];

            llSay(0,(string)responsedata["translatedText"]);
        }
}

public void default_event_listen(LSL_Types.LSLInteger channelIn, LSL_Types.LSLString name, LSL_Types.LSLString id, LSL_Types.LSLString message)
{
    if(channelIn==1)
    {
        string toTranslate = (string) message;
        requestID = llHTTPRequest( "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&q="+toTranslate+"&langpair="+sourceLang+"%7C"+targetLang, new LSL_Types.list(), "" );  

    }
    else if(channelIn==2)
    {
        sourceLang = (string) message;
    }
    else if(channelIn==3)
    {
        targetLang = (string)message;
    }
    else if(channelIn==4)
    {
        llOwnerSay("LANGUAGE (CODE)");
        llOwnerSay("*  Arabic (ar)");
        llOwnerSay("* Bulgarian (bg)");
        llOwnerSay("* Chinese (zh)");
        llOwnerSay("* Croatian (hr)");
        llOwnerSay("* Czech (cs)");
        llOwnerSay("* Danish (da)");
        llOwnerSay("* Dutch (nl)");
        llOwnerSay("* English (en)");
        llOwnerSay("* Finnish (fi)");
        llOwnerSay("* French (fr)");
        llOwnerSay("* German (de)");
        llOwnerSay("* Greek (el)");
        llOwnerSay("* Hindi (hi)");
        llOwnerSay("* Italian (it)");
        llOwnerSay("* Japanese (ja)");
        llOwnerSay("* Korean (ko)");
        llOwnerSay("* Norwegian(no) ");
        llOwnerSay("* Polish (pl)");
        llOwnerSay("* Portuguese (pt-PT)");
        llOwnerSay("* Romanian (ro)");
        llOwnerSay("* Russian (ru)");
        llOwnerSay("* Spanish (es)");
        llOwnerSay("* Swedish (sv)"); 

    }
}