Hello,

> Yeah, that would be pretty cool. Then all you need
> is a program that can load those monster data sets.
;)

Hardware is the problem here. L3DTVi2 can load big
mosaic maps (I’ve looked at 32k x 32k maps before),
but you still have to keep your horizon-distance
relatively short because there isn’t enough RAM to
view any further. Fancier algorithms might help (eg.
using pre-calculated low-resolution meshes for LOD,
etc), but ultimately you are always going to be
limited by how much RAM you’ve got. Of course, clever
procedurals have no RAM-problems, but they are
effectively just offloading the problem to the CPU, so
again you’ve got a hardware-limited problem.

> Unfortunately for us that's just the very first
stage.
> That's our starting point. It just gets worse from
there, hehe.
> The 20k map is I believe 200 meters per pixel. We
want to cut
> that down to about 10 eventually.

Other than the human-time involved in editing a
monster like that, this size shouldn’t be a serious
problem. By the time you get to that resolution a
couple of hundred TB won’t be a lot of disk-space.
L3DT can already handle maps that large (the texture
size-limit is 2M x 2M, and that can be increased to
about 1G x 1G if required). The SDK will also handle
terapixel maps out-of-the-box.

> That's kind of a non-ideal solution. It would
probably result
> in pretty harsh lines if a larger area was not
edited…A better
> solution is needed I think.

I might have another try at explaining this, if you
don’t mind.

I think you’ll have trouble with editing map tiles
piece-wise and gluing them together afterwards. That
is a hard problem. What I propose is that you start
off with a really coarse map, like 50 x 50, that
covers the whole world. Something rough that could be
bashed-together in a couple of hours.

Then, using an interpolation algorithm, inflate the
whole thing by, say, 100x (splitting it into tiles if
required). Now, you can refine the heightmap using
some manual tools (eg. Wilbur), or whatever procedural
overlays you have at hand. Since you’re editing a
heightmap that is already continuous and seam-free,
there won’t be any seam problems unless you do
something to one tile that doesn’t ‘leak across’ to
the neighbouring tiles. The method I described
previously is a way to prevent this.

Anyhow, once everyone involved has finished refining
their patches of the heightmap, inflate the bugger
again and repeat.

This method is what L3DT uses to generate heightmaps,
although the inflation is usually at steps of 2x and
all the editing/effects are handled automagically. And
it works on large maps too, without seams, since the
effects and overlays do indeed leak across tiles
appropriately. Anyhoo, there is no technical reason
that this method can’t be modified to use larger
inflation steps with manual editing in-between.

Did that make any sense, or am I still talking crap?

> Or some sort of overlap system perhaps? Ultimately
we want to
> have a bunch of equal tiles that can be downloaded
and glued
> together to form one contiguous terrain covering any
area you
> want, without seams. So the seam issue is critical.

It is much harder to fix seams than it is to prevent
them from forming. You need an editing strategy that
forbids the formation of seams, as the above method
does (I think, unless I’ve missed something).

> I'll tell you right here ideally we'd like to get
several
> developers involved, brainstorming on how to address
these
> problems. We figure if the software can be made free
or open
> source or something, it'll probably be applicable to
other
> people's projects too.

I’m happy to brainstorm ideas, but I can’t promise
that I’ll be able to write any huge amount of code.
We’ll see, I guess.

> So eventually it'd be great if you and Ray of
Leveller and
> Stephen of World Machine and the Wilbur guy
(forgetting his
> name right now) all got together and made us a kick
ass
> app/system/framework/dingly. And then I woke up from
> my beautiful dream. ;)

I think that may stay just a dream, to be honest. I
can’t speak for all the developers you mentioned; but
I’m pretty sure it would not be in our best interest
to produce one product that could fundamentally
undermine the markets for our individual projects.
Making it open-source would only make matters worse,
in a purely selfish sense, because we would be
spending a lot of time, donating a lot of IP, and
profiting very little. I simply can’t see the benefits
(to us, that is).

Whoa, sorry for that bitter, mercenary cynicism. I
think I need a lie-down after a long week or
something.

Anyhoo, a one-package approach may not be necessary if
all the tools you mentioned were more interoperable,
used common formats, etc. I’m happy to work towards
this goal.

Cheers,
Aaron.


--- Oshyan Greene <oshyan@...> wrote:

> > -----Original Message-----
> > From: L3DT_users_group@yahoogroups.com
> > [mailto:L3DT_users_group@yahoogroups.com] On
> Behalf Of Aaron Torpy
> > Sent: Tuesday, November 15, 2005 11:03 PM
> > To: L3DT_users_group@yahoogroups.com
> > Subject: RE: [L3DT users' group] SDK progress
> report
> >
> > Sounds like the network stuff might be very handy
> > > for the Middle Earth DEM Project.
> >
> > I hope so...ME-DEM was one of the projects I had
> in
> > mind when I proposed the network stuff
>
> Yay! <G>
>
> > (the other main
> > one being Beowulf clustering of map calculations,
> for
> > huge maps in a hurry, and for the 'cool' factor).
>
> Yeah, that would be pretty cool. Then all you need
> is a program that can
> load those monster data sets. ;)
>
> > For the benefit of the group-members not familiar
> with
> > ME-DEM (http://me-dem.ashundar.com), it is an
> > ambitious project to create a digital elevation
> map of
> > 'Middle Earth', from J.R.R. Tolkien's Lord of The
> > Rings, et. al. At my last reading, the goal is to
> make
> > a 20k x 20k pixel final heightmap.
>
> Unfortunately for us that's just the very first
> stage. That's our starting
> point. It just gets worse from there, hehe. The 20k
> map is I believe 200
> meters per pixel. We want to cut that down to about
> 10 eventually. So once
> we have the 20k map, derived from hand-drawn maps of
> various sources (and
> covering only a portion of Middle Earth mind you -
> about 1/4 I think), we
> divide up into 100x100 tiles (I think - or maybe
> 200x200) and scale by 10
> times to get 1000x1000 pixel terrains. Those
> terrains are then refined by
> hand to get more realistic detail at 20m/pixel. Then
> we divide again to get
> the 10m (or better) data and refine again by hand. I
> forget the actual
> numbers but we'll actually end up with literally
> 10's of thousands of
> terrain tiles of 1000x1000 or something. It's pretty
> scary. And that's just
> for 1/4 (or less) of the whole world. :p
>
> > Since the
> > terrain-shaping has to be (at-least partially)
> manual
> > to get the right result, the project will require
> an
> > awful-lot of person-hours to build a map that
> large.
>
> Yeah, multiply "awful-lot" by 1000 given the above.
> <G>
>
> > One of the ideas that's been proposed to help-out
> is
> > the creation of a distributed server/client system
> so
> > that individual contributors can chip away at
> small
> > areas and send back the results without causing
> nasty
> > duplication or edge mismatch problems.
>
> Indeed. Another thing that would help is if we could
> load up our base maps
> in L3DT and use an extension of the current terrain
> definition system to add
> semi-directed procedural detail to our base
> terrains. We're getting closer
> to that now as L3DT gains capability.
>
> > > If we could utilize this to setup a master file
> list
> > on a
> > > server and have a check-in/check-out system,
> simple
> > map-based
> > > GUI tile selection with automated tile
> positioning
> > after edit
> > > and upon export, and perhaps even some way to
> match
> > up
> > > tile edges... well, it'd help us out a
> helluvalot.
> >
> > Hmm...I think I can see how all that would work.
> The
> > tile-edge matching could be tricky, but I guess we
> > could always forbid the editing of a 1px-thick
> border
> > of the downloaded area (if you want to edit those
> > bits, download the surrounding tiles too).
>
> That's kind of a non-ideal solution. It would
> probably result in pretty
> harsh lines if a larger area was not edited. And
> ultimately there will
> always be tile edges, whether you're working with 4
> 1000 pixel tiles (so the
> 4 interior seams are covered, but there are still 4
> edge seams), or a single
> 1000 pixel tile. A better solution is needed I
> think.
>
> > Then-again,
> > there's always the option of linear (or
> non-linear)
> > blending of overlapping tile edges. I'd rather
> avoid
> > this 2nd option, though, since it would add
> > complications to the server.
>
> It may be necessary given the above. Or some sort of
> overlap system perhaps?
> Ultimately we want to have a bunch of equal tiles
> that can be downloaded and
> glued together to form one contiguous terrain
> covering any area you want,
> without seams. So the seam issue is critical.
>
> > As for the other stuff, it's conceptually pretty
> neat
> > 'n' tidy. All the network socket code would go
> into a
> > common plugin. The map server could be a
> stand-alone
> > program independent of L3DT, but using the SDK and
> > relevant plugins. The client-side would preferably
> be
> > a plugin for L3DT, but could initially be a
> > stand-alone job too (just to keep it simple).
> > Firewalls/NATs/etc can be worried about later.
>
> Good, good...
>
> > Anyhow, it's a good think that ME-DEM is a
> long-term
> > project, because there's an awful lot of coding to
> > make that stuff work.
>
> Yeah, loooong term. We're definitely on the long
> track. <G>
>
> > I'll start playing around with
> > it early/mid next year, hopefully once I've got a
> nice
> > beta-release of the SDK going.
>
> Sounds great to me! I'm looking forward to it. :)
>
> I'll tell you right here ideally we'd like to get
> several developers
> involved, brainstorming on how to address these
> problems. We figure if the
> software can be made free or open source or
> something, it'll probably be
> applicable to other people's projects too. Maybe
> even commercial endeavors.
> Or it can at least be made general enough that that
> would be the case. So
> eventually it'd be great if you and Ray of Leveller
> and Stephen of World
> Machine and the Wilbur guy (forgetting his name
> right now) all got together
> and made us a kick ass app/system/framework/dingly.
> And then I woke up from
> my beautiful dream. ;)
>
> - Oshyan
>
>