OpenSim turned three years old yesterday. In the past three years, we’ve been watching as the initial germ of an idea evolved and changed into the current platform, which is poised to go even further.
Fashion Research Institute has a unique view of the evolution of this open source virtual world platform. We are designers; users and consumers of OpenSim. We have to wait for the developers and the coders to make the platform so we can do something with it. We have a tech team, so sometimes things we want we don’t have to wait so long for our wishes as other users, but mostly our work in OpenSim has been about using it, breaking it, finding the issues, and reporting them back to the Core developers who can then fix what we have found which is broken.
In the 2 1/2 years we’ve been working on the platform, we’ve come from that initial sense of awe and wonderment at being in a new world running on our desktops; one that was unformed and simple: Ruth on a Dot. Ruth on a Dot wasn’t itself terribly exciting, but the sense of futurity, the sense of possibility, sweeping vision for what the platform could become was simply breathtaking. That was how it felt back in September 2007, when we brought up our first dot of land in our first OpenSim instance. Nothing worked quite right – there was no persistence, no physics, really not much to commend it, except for the fact you could create objects…and the vision of what it could become. That is what freedom looked like in September 2007: Ruth on a Dot. What it felt like was a path opening before us.
The path has led us to some amazing experiences. In 2008, in our research collaboration with IBM, we built Shengri La Spirit in a matter of days, on a platform that wasn’t really ready for what we wanted it do. But we persevered, the platform cooperated enough, and Spirit was built. We almost lost Shengri La Spirit – in fact, it was lost, for more than a year. But, and this is critical to understanding why a platform like OpenSim is so important and needed: after a year of that data lying in a heap, one of our team members was able to resurrect Spirit. It wasn’t lost, as so much of our work in Second Life has been lost.
Now Spirit is hosted on the ScienceSim grid as part of our research collaboration with Intel Labs. Anyone can log in, and walk through the trellis hall. Anyone can see the prim drift and shift that occurred as we fought, cajoled and encouraged something that wasn’t quite ready to do what we wanted it to do. There’s ample visual evidence of where the state of OpenSim was; we have never cleaned the build up since we feel it is a matter of historical record. We have the data in an OAR file. It is ours to do with as we please. But that is where we were, in 2008.
In 2009, we began a research collaboration with Intel Labs to focus on performance and content issues. We provided content, including large-scale, complicated regions, which the Intel engineers used as workloads and test cases. Our development of the 256,000 object region Shengri La Chamomile resulted in several performance patches contributed back to the OpenSim code base. Chamomile, too, serves as a benchmark: this is where the OpenSim codebase was in late 2009. It too, can be visited – however, the experience is challenging to anyone with less than robust hardware and bandwidth; and the viewers aren’t really up to rendering the scene. Chamomile stands as a challenge for viewer developers, and marks the state of development of OpenSim in 2009.
Now, in 2010, we’d like to showcase the first of what we expect to be many developments in 2010. For the past month, the team in ScienceSim has been head’s down building infrastructure designed to support our future work with OpenSim. We are developing content standards that govern how collections of content can be managed, curated and tracked. We have been developing tools that are useful for the audience of educators and enterprise using the ScienceSim grid. We have been defining policy and procedure (and documenting it as we go) to define best practices in settling new OpenSim-based grids for enterprise and education. We are creating, building, enhancing, and evolving all of this on top of the OpenSim code base, which is now robust enough to support the work we’re doing.
We are now standing on the threshold of something amazing, which has the same feeling of being an early OpenSim Ruth on a Dot in 2007.
The following images were snapped in ScienceSim grid yesterday, shortly after the regions had the terrain for the Yellowstone National Park loaded. The terrain data was provided by Dr. Brian Quinn, whose work with translating LiDAR data into Second Life and OpenSim terrain is well known by many. Brian provided the terrain at 1:10.38 scale, and it is loaded into a series of megaregions that stretch on as far as the viewer can see.
This is an included post about these regions from Mic Bowman, Intel’s lead engineer on this project, on Intel’s involvement with ScienceSim:
“As part of sizing the hw requirements for a mirror world project we’re exploring… we wanted to do some scalability tests on the capacity of individual simulators in terms of the total number of regions. we wanted baseline numbers that focus on just the most simple region configuration: a completely empty region with default terrain. That is… this is JUST simulator overhead.
All tests are done on one of our blades… dual proc, quad core with 16G ram running 64bit ubuntu. mono 2.6. and the tests are hosting 1024 regions in a 32×32 grid. The simulator configuration was our standard sciencesim config (XEngine, ODE, groups, wind, sun, etc).
The first configuration ran all 1024 regions in *one* simulator. I honestly figured this would crash quickly. It didn’t. We managed to get all 1024 regions created and running well enough to walk around.It did take almost 8 hours to start. The first couple regions were created in 1-2 seconds each. by the end, it was taking 4-5 minutes per region. Clearly there is something quadratic in there (stop using linear lists, they are evil! :-). But it could have been the mono garbage collector. who knows… not sure its worth too much investigation because I can’t imagine ever running a config like this for real. The simulator did crash when we opened the map in the viewer. The crash was caused because we ran out of sockets. While it was running, the simulator used just over 10G of ram and was running at about 700% CPU utilization (kind of scary to see load averages in the 900 range!).
The second configuration ran 16 simulators each with 64 regions. Startup took about 30 minutes (each of the 16 simulators avoided the quadratic “knee” we hit with the one big simulator). Consumed about 11G of memory and was again consuming essentially all cycles on the machine (completely idle regions aren’t very idle). All 16 simulators died just after startup with a “too many open files” error. Not sure what caused it, but all of them died loading the same terrain dll as part of some wildcard function looking for dlls. That one is an interesting bug.
The final configuration and the one we’re shooting for in the short term runs 16 simulators, each with an 8×8 megaregion. again startup was around 40 minutes. we might be able to cut that time down by starting all 16 simulators simultaneously rather than 4 at a time. I really just wanted to make sure that some of the spikes we see in startup didn’t cause failures (some race condition causes all threads to consume maximum processor cycles randomly on startup right now). and, well… it just worked. I figured the viewer would die horribly (it can’t handle 250K “real” prims very well) but it survived just fine. turn off far clip with 8×8 megaregions providing neighbors and you are “capable” of contacting simulators in a 24×24 region range!. the view is pretty cool though it seems to not go beyond 12×12. 🙂 there are a bunch of problems (like you can’t see the terrain in the region you’re standing in)… but there are a LOT of little humps of terrain in view. Oh… and it takes a lot of patience to get everything loaded.
So… the conclusion… this opensim thing is pretty amazing! Good work everyone.
OpenSim’s fourth year builds on the work of the past three years. We start 2010 with an OpenSim benchmark that shows how far it has come from 2007. What was accomplished last week has sweeping implications for enterprise application on a very large scale. It has implications for the use of OpenSim for modeling an array of different sorts of data. What was so breathtaking about standing in the Yellowstone terrain and looking around was the vision of how such a creation could be used: for geology education; for management of land, water, and mineral rights; showcasing real estate development; managing a national park in real time; modeling war games; the development of a created virtual space that could be used in common for all of these uses by an array of users. All of these use cases are possible. And they all start here, with a single sweeping expanse of virtual land that models a part of the real world, and which was instantiated in the space of less than a week.