Discussion:
[CsMain] Conversion of Quake3 maps
(too old to reply)
Thomas Hieber
1999-12-30 19:36:15 UTC
Permalink
Hello,

I finally finished map2cs to make it able to completely convert Quake 3 map
files. This also includes Curved Surfaces. Despite of what I feared, they
look pretty well in Crystal Space. I emulate (?) the Quake 3 behaviour by
splitting of Quake 3 curves into smaller Bezier patches. I was afraid, that
there would be sharp edges, where the single Beziers meet, but you won't
notice anything like that.
So much for the good news.
I did some tests with the converted Q3dm1sample.map. (One of the maps, that
was already in the demo, and that contains the famous mouth and tongue) This
map will show, that there is still some way to go, untill we can compete
with Quake3, even if we limit comparison only to rendering and don't talk
about support for games.
My major problems were:
- Runtime speed. I get about 5fps in bad places of the map (Without curved
surfaces and with C-buffer/ Coverage tree). The original Q3 runs never
slower than 20fps in the same place on the same hardware. (Both are using
OpenGL rendering btw.)
- Startup speed. Crystal Space takes ages to calculate lighting in that map
(Shining lights...). For tests, I would like to have an option to use some
sort of sloppy and fast lighting. (No shadows for example)
- Even when the lightmaps are stored in the map, startup still takes very
long (also in Shining lights...).
- Quality of the resulting lighting is really bad, compared to Quake 3. (no
radiosity...)
- Curved surfaces don't have proper lighting. They show completely different
shadows and lights than the regular walls near them. They still look good
enough for themselfes, but when they are attached to a wall, you will see,
where the Curved surface starts, and where it ends. Maybe we should use
lightmaps for Curves Surfaces too.
- I can't use advanced visibility like c-buffer or coverage tree, because
they all require statbsp, and you can't use curved surfaces in these
sectors. (They just don't show.)
- CS still wastes fillrate, because it will paint all sector walls with
z-buffer and texture. There is still no option in the map format, to draw a
polygon only in the z-buffer, without changing screen pixels. (Quake 3 has a
special texture for this called "common/chalk")

Anyway, I will add the source changed to map2cs to CVS soon. I will also try
to make some screenshots, so Jorrit can put them on the homepage.

Thomas
jorrit
1999-12-30 19:47:57 UTC
Permalink
Post by Thomas Hieber
Hello,
- Runtime speed. I get about 5fps in bad places of the map (Without curved
surfaces and with C-buffer/ Coverage tree). The original Q3 runs never
slower than 20fps in the same place on the same hardware. (Both are using
OpenGL rendering btw.)
This is caused by the curved surfaces even though you don't see them. Curved
surfaces are always rendered. They are not included in the visibility algorithm

for the c-buffer. That's something that still needs to be done.
Post by Thomas Hieber
- Startup speed. Crystal Space takes ages to calculate lighting in that map
(Shining lights...). For tests, I would like to have an option to use some
sort of sloppy and fast lighting. (No shadows for example)
This is caused by the curved surfaces. Lighting those is extremely slow right
now.
Post by Thomas Hieber
- Even when the lightmaps are stored in the map, startup still takes very
long (also in Shining lights...).
Curved surfaces again :-)
Post by Thomas Hieber
- Quality of the resulting lighting is really bad, compared to Quake 3. (no
radiosity...)
That's an old problem.
Post by Thomas Hieber
- Curved surfaces don't have proper lighting. They show completely different
shadows and lights than the regular walls near them. They still look good
enough for themselfes, but when they are attached to a wall, you will see,
where the Curved surface starts, and where it ends. Maybe we should use
lightmaps for Curves Surfaces too.
There are no shadows on curved surfaces and curved surfaces don't
cast shadows. This causes the problems you are seeing.


Curved surfaces need a lot of work. It is unfortunate that the original author
(Ayal Pinkus) ran out of time.

Greetings,
DrEvil
1999-12-30 22:52:57 UTC
Permalink
Post by jorrit
Post by Thomas Hieber
- Curved surfaces don't have proper lighting. They show completely different
shadows and lights than the regular walls near them. They still look good
enough for themselfes, but when they are attached to a wall, you will see,
where the Curved surface starts, and where it ends. Maybe we should use
lightmaps for Curves Surfaces too.
There are no shadows on curved surfaces and curved surfaces don't
cast shadows. This causes the problems you are seeing.
And on a side note, Quake3 doesn't support curved surfaces casting shadows
either :) Curved surfaces also have no VIS blockage. That's what the caulk
brushes in Q3 are for. Caulk brushes also help correct lighting in Q3 :) So,
CS not doing this is just like Q3's limitations :) Although HINT brushes in
Q3 block limit the drawing of normal brushes and curves, and some entities.
(I don't know if all are effected by it)

-EvilTypeGuy
***@qeradiant.com
Thomas Hieber
1999-12-30 21:12:18 UTC
Permalink
Post by Thomas Hieber
Post by jorrit
Post by Thomas Hieber
- Curved surfaces don't have proper lighting. They show completely
different
Post by jorrit
Post by Thomas Hieber
shadows and lights than the regular walls near them. They still look
good
Post by jorrit
Post by Thomas Hieber
enough for themselfes, but when they are attached to a wall, you will
see,
Post by jorrit
Post by Thomas Hieber
where the Curved surface starts, and where it ends. Maybe we should use
lightmaps for Curves Surfaces too.
There are no shadows on curved surfaces and curved surfaces don't
cast shadows. This causes the problems you are seeing.
And on a side note, Quake3 doesn't support curved surfaces casting shadows
either :) Curved surfaces also have no VIS blockage. That's what the caulk
brushes in Q3 are for. Caulk brushes also help correct lighting in Q3 :) So,
CS not doing this is just like Q3's limitations :)
Ok, then this means, we need no revolutionary new technique, we just need
finetuning. (At least we should see no visual problems, where there are no
visual problems in Q3)

Thomas
Thomas Hieber
1999-12-30 20:43:03 UTC
Permalink
Post by jorrit
Post by Thomas Hieber
- Runtime speed. I get about 5fps in bad places of the map (Without curved
surfaces and with C-buffer/ Coverage tree). The original Q3 runs never
slower than 20fps in the same place on the same hardware. (Both are using
OpenGL rendering btw.)
This is caused by the curved surfaces even though you don't see them. Curved
surfaces are always rendered. They are not included in the visibility algorithm
for the c-buffer. That's something that still needs to be done.
Not exactly. This problem even happens, when I remove any curved surfaces
from the map file. IMHO, this just shows, that Quake 3 is very well
finetuned, even for maps with high polygon counts.
Post by jorrit
Curved surfaces need a lot of work. It is unfortunate that the original author
(Ayal Pinkus) ran out of time.
Indeed. I hope somebody else will take a look at it.

btw: I am still waiting for my testmap to start up. I started walktest
(release mode) at 19:15 and now its 21:45 and I just crossed "Shining
lights, 65%". I used to run my tests with only a fraction of the map. (And I
already switched of high-quality lightmaps)


Thomas
Eric Sunshine
1999-12-30 22:27:17 UTC
Permalink
The official Crystal Space IRC forum is moving to
irc.linux.com:6667 #CrystalSpace. (The old forum was hosted at
irc.icq.com.) irc.linux.com welcomes open projects such as Crystal
Space and appears to be better maintained (more reliable services)
than irc.icq.com. (In fact, the Source Forge team also meets on
irc.linux.com at #sourceforge.)

Please begin using the new server at irc.linux.com from this point
onward since the old one at irc.icq.com is now deprecated. Jorrit
will be updating the web page to point at the new server just as
soon as he gets back to the office.

-- ES
jorrit
1999-12-31 19:24:31 UTC
Permalink
Post by Thomas Hieber
Post by jorrit
Post by Thomas Hieber
- Runtime speed. I get about 5fps in bad places of the map (Without
curved
Post by jorrit
Post by Thomas Hieber
surfaces and with C-buffer/ Coverage tree). The original Q3 runs never
slower than 20fps in the same place on the same hardware. (Both are
using
Post by jorrit
Post by Thomas Hieber
OpenGL rendering btw.)
This is caused by the curved surfaces even though you don't see them.
Curved
Post by jorrit
surfaces are always rendered. They are not included in the visibility
algorithm
Post by jorrit
for the c-buffer. That's something that still needs to be done.
Not exactly. This problem even happens, when I remove any curved surfaces
from the map file. IMHO, this just shows, that Quake 3 is very well
finetuned, even for maps with high polygon counts.
Hmm, I'd like to investigate that map. Maybe I can find out what the
performance problems are. Profiling CS while running that map will be
interesting
at least.
Post by Thomas Hieber
btw: I am still waiting for my testmap to start up. I started walktest
(release mode) at 19:15 and now its 21:45 and I just crossed "Shining
lights, 65%". I used to run my tests with only a fraction of the map. (And I
already switched of high-quality lightmaps)
High-quality lightmaps don't increase lighting time very much. The curved
surfaces are probably the cause.

Greetings,
Robert S. Elsner
1999-12-30 19:48:04 UTC
Permalink
http://www.planetquake.com/aftershock

A Quake3:Arena compatible renderer. GPL, but it might provide ideas.

It seems to work *very* well.

-r
Seth Galbraith
1999-12-30 21:23:23 UTC
Permalink
Post by DrEvil
And on a side note, Quake3 doesn't support curved surfaces casting
shadows either :)
But does it support shadows being cast on curved surfaces? This is
important to keep the lighting consistent between flat and curved
surfaces.
Post by DrEvil
Curved surfaces also have no VIS blockage. That's what the caulk
brushes in Q3 are for. Caulk brushes also help correct lighting in Q3
Note to Thomas: the texture on these brushes is called "common/caulk",
not "common/chalk" :-)
Post by DrEvil
Not exactly. This problem even happens, when I remove any curved
surfaces from the map file. IMHO, this just shows, that Quake 3 is
very well finetuned, even for maps with high polygon counts.
It is very important to keep this in mind:

Quake 3 uses a CSG technique to build maps. It portalizes maps and
removes hidden surfaces. Map2CS puts all of the polyhedrons used to
create the map into a single sector and relies on the Crystal Space
Octree/BSP system for visibility.

By no stretch of the imagination is this an optimal way to convert these
maps! The Octree system was designed for very different types of maps.
Quake style maps are exactly the kinds of maps that the Portal algorithm
is designed for. So unless these maps are converted into maps with many
sectors and portals with all of the hidden surfaces removed, they will
continue to run very slowly.

So adding new visibility algorithms and adding new ones are great ideas,
but it's also important to use the right algorithm in the right situation
:-)

There are two ways to improve Quake-style indoor map conversion:

1. Add hidden surface removal and sector/portal generation to map2cs
2. Revive the Quake2CS compiled .bsp format Quake level convertor.
__ __ _ _ __ __
_/ \__/ \__/ Seth Galbraith "The Serpent Lord" \__/ \__/ \_
\__/ \__/ \_ ***@krl.org #2244199 on ICQ _/ \__/ \__/
_/ \__/ \__/ http://www.planetquake.com/simitar \__/ \__/ \_
Thomas Hieber
1999-12-30 23:38:49 UTC
Permalink
Post by Seth Galbraith
Post by DrEvil
Curved surfaces also have no VIS blockage. That's what the caulk
brushes in Q3 are for. Caulk brushes also help correct lighting in Q3
Note to Thomas: the texture on these brushes is called "common/caulk",
not "common/chalk" :-)
I didn't look it up in my dictionary, but it was some word with a 'c', 'l'
and a 'k'. :-)
Post by Seth Galbraith
Post by DrEvil
Not exactly. This problem even happens, when I remove any curved
surfaces from the map file. IMHO, this just shows, that Quake 3 is
very well finetuned, even for maps with high polygon counts.
Quake 3 uses a CSG technique to build maps. It portalizes maps and
removes hidden surfaces. Map2CS puts all of the polyhedrons used to
create the map into a single sector and relies on the Crystal Space
Octree/BSP system for visibility.
By no stretch of the imagination is this an optimal way to convert these
maps! The Octree system was designed for very different types of maps.
Quake style maps are exactly the kinds of maps that the Portal algorithm
is designed for. So unless these maps are converted into maps with many
sectors and portals with all of the hidden surfaces removed, they will
continue to run very slowly.
So adding new visibility algorithms and adding new ones are great ideas,
but it's also important to use the right algorithm in the right situation
:-)
1. Add hidden surface removal and sector/portal generation to map2cs
Maybe I forgot to mention, but map2cs can already remove hidden surfaces.
(You need to add "removehidden = 1" in the "[general]" section in map2cs.cfg
and have some patience when converting.) And as far as I know, Quake's
"portalisation" is just the building of a BSP tree. Of course, when you have
finished building a BSP, you will end up with a huge bunch of convex sectors
and portals between them. Thats not _very_ different, from the way CS
behaves, when you add a STATBSP() to a sector. It's just, that CS will build
be BSP. (Plus a large-scale Octree)
I guess the main performance hit is, because Quake3 is very optimised to get
ultimate performance from the OpenGL interface. CS is not really
architecturally optimised. That was not needed for a long time, because when
using the software renderer, the scanline drawing functions were the
bottleneck, and OpenGL was way faster without putting much effort into it.
If you were to design an engine that provides ultimate performance for
hardware accelerated rendering (and I am _not_ talking about T&L _here_),
then you need to take into account the overhead involved in every single
call to the renderer and the penalties involved for every context switch in
the renderer. (There are lots of context switches: CS switches the context
at least twice for every single polygon it renders on screen.) I am sure Q3
takes these problems into account.
Post by Seth Galbraith
2. Revive the Quake2CS compiled .bsp format Quake level convertor.
I would not expect better performace from that. And I don't like the idea of
relying on id's tools there. Map is a simple format, you could easily edit
by hand if you like. BSP is pretty obfuscated.

Thomas
Peter Donald
1999-12-31 05:50:11 UTC
Permalink
Hiya,
Post by Seth Galbraith
It seems to me that CS and Quake3 have very different visibility systems
(Octree/BSP versus BSP/PVS?) I wonder how much of the performance
difference is optimization and how much is because of the algorithm?
The PVS is one of the major reasons that the Quake style engines run so
fast compared to similar engines. They trade a lot of time to compile map
(I just vised a map for about 33 hours) for speed at execution time. They
also have relatively small buckets so the overdraw is minimised without the
cost of all the fancty 2-d clipping algorithms. If there was a fast way to
generate visibility info at load time with small enough sector/cells then
you could in theory remove all the 2-d clipping.

The problem is balancing the cost of overdraw vs vis time and also
incrementely doing vis-es. I will be trying in the next couple of weeks to
get this happening so stay tuned.

The vis also has a problem in worlds you want to be dynamic. Everytime a
new cell/portal is added the visdata has to be recalcd. Thou it may be
possible to somehow have different size buckets/cells/sectors that handle
vis slightly differently (ie only vis betweeen cells for octreee/roam
worlds .. complete vis for bsp worlds etc)

Pete
Void
2000-01-01 22:12:39 UTC
Permalink
Post by Peter Donald
Hiya,
Post by Seth Galbraith
It seems to me that CS and Quake3 have very different visibility systems
(Octree/BSP versus BSP/PVS?) I wonder how much of the performance
difference is optimization and how much is because of the algorithm?
The PVS is one of the major reasons that the Quake style engines run so
fast compared to similar engines.
I would really like it if somebody felt like explaining to me a little
on what PVS is.
Peter Donald
2000-01-02 02:16:25 UTC
Permalink
Hiya,
Post by Void
I would really like it if somebody felt like explaining to me a little
on what PVS is.
PVS == Potentially Visible Set

So what this means is that when you are drawing the world your eyes are
located in a volume somewhere (lets call it a bucket). This bucket has a
list of other buckets it is possible to see from it. In the rendering loop
you basically choose all geometry that is visible from the current geometry
clip it against the view frustrum and draw it all. This has advantages in
that it is very fast at runtime, it is ideal for hardware orientated apis
where a little overdraw is not a problem.

The tradeoff of this method is the size of the bucket vs the cost in
storing and generating PVS. Quake games take a while to generate the PVS
data (I just compiled for 36 hours on a p3-500) and can only reduce cost of
storage by compressing PVS data. However Quake has relatively small buckets
and thus reduces overdraw considerably.

There is a couple of places on the web you can find information on this
.... I think there is a researcher by the name Seth Teller that wrote
something like Determining occluded geomtry in densly occluded worlds.


Pete
Peter Donald
2000-01-04 22:56:58 UTC
Permalink
Hiya,
Post by Seth Galbraith
Post by Peter Donald
The vis also has a problem in worlds you want to be dynamic. Everytime
a new cell/portal is added the visdata has to be recalcd. Thou it may
be possible to somehow have different size buckets/cells/sectors that
handle vis slightly differently (ie only vis betweeen cells for
octreee/roam worlds .. complete vis for bsp worlds etc)
Couldn't you apply the VIS techniques based on sectors rather than whole
worlds? (i.e. complete VIS for bsp SECTORS?)
It would be a good thing. You could do all sorts of things like load a
quake binary directly as a sector and use all its data etc. However last
time I mentioned this Jorrit said he wanted to be able to keep everything
dynamic.

Pete
Jorrit Tyberghein
2000-01-05 06:59:14 UTC
Permalink
Post by Peter Donald
Hiya,
Post by Seth Galbraith
Post by Peter Donald
The vis also has a problem in worlds you want to be dynamic. Everytime
a new cell/portal is added the visdata has to be recalcd. Thou it may
be possible to somehow have different size buckets/cells/sectors that
handle vis slightly differently (ie only vis betweeen cells for
octreee/roam worlds .. complete vis for bsp worlds etc)
Couldn't you apply the VIS techniques based on sectors rather than whole
worlds? (i.e. complete VIS for bsp SECTORS?)
It would be a good thing. You could do all sorts of things like load a
quake binary directly as a sector and use all its data etc. However last
time I mentioned this Jorrit said he wanted to be able to keep everything
dynamic.
I said I first want to concentrate on algorithms that are better applicable for
dynamic worlds. Then I want to move on to really static algos.


Greetings,

--
==============================================================================
***@uz.kuleuven.ac.be, University Hospitals KU Leuven BELGIUM

Bigmac's brother was reliably believed to be in the job of moving video
recorders around in an informal way.
-- (Terry Pratchett, Only You Can Save Mankind)
==============================================================================
Seth Galbraith
1999-12-31 00:00:43 UTC
Permalink
And as far as I know, Quake's "portalisation" is just the building of
a BSP tree. Of course, when you have finished building a BSP, you will
end up with a huge bunch of convex sectors and portals between them.
Thats not _very_ different, from the way CS behaves, when you add a
STATBSP() to a sector. It's just, that CS will build be BSP. (Plus a
large-scale Octree)
That's good to know. I was wondering how Quake ended up with sectors and
portals even though it isn't a "portal engine". Now I think I understand
the relationship a little better.

It seems to me that CS and Quake3 have very different visibility systems
(Octree/BSP versus BSP/PVS?) I wonder how much of the performance
difference is optimization and how much is because of the algorithm?

(Of course I have no doubt that focusing on one rendering API from the
early years of Quake development is a major factor in the speed of Q3:A)
__ __ _ _ __ __
_/ \__/ \__/ Seth Galbraith "The Serpent Lord" \__/ \__/ \_
\__/ \__/ \_ ***@krl.org #2244199 on ICQ _/ \__/ \__/
_/ \__/ \__/ http://www.planetquake.com/simitar \__/ \__/ \_
Seth Galbraith
2000-01-04 19:40:04 UTC
Permalink
Post by Peter Donald
The PVS is one of the major reasons that the Quake style engines run
so fast compared to similar engines. They trade a lot of time to
compile map (I just vised a map for about 33 hours) for speed at
execution time.
Spending a few days PVS-optimizing a map that took weeks to develop so
that it runs several times faster is a worthwhile project - especially if
the PVS optimizing is running during idle time (i.e. overnight) or on a
seperate machine.
Post by Peter Donald
The vis also has a problem in worlds you want to be dynamic. Everytime
a new cell/portal is added the visdata has to be recalcd. Thou it may
be possible to somehow have different size buckets/cells/sectors that
handle vis slightly differently (ie only vis betweeen cells for
octreee/roam worlds .. complete vis for bsp worlds etc)
Couldn't you apply the VIS techniques based on sectors rather than whole
worlds? (i.e. complete VIS for bsp SECTORS?)
__ __ _ _ __ __
_/ \__/ \__/ Seth Galbraith "The Serpent Lord" \__/ \__/ \_
\__/ \__/ \_ ***@krl.org #2244199 on ICQ _/ \__/ \__/
_/ \__/ \__/ http://www.planetquake.com/simitar \__/ \__/ \_
Seth Galbraith
2000-01-05 02:14:17 UTC
Permalink
Post by Peter Donald
It would be a good thing. You could do all sorts of things like load a
quake binary directly as a sector and use all its data etc. However
last time I mentioned this Jorrit said he wanted to be able to keep
everything dynamic.
Keeping everything dynamic when you don't need to is stupid.
__ __ _ _ __ __
_/ \__/ \__/ Seth Galbraith "The Serpent Lord" \__/ \__/ \_
\__/ \__/ \_ ***@krl.org #2244199 on ICQ _/ \__/ \__/
_/ \__/ \__/ http://www.planetquake.com/simitar \__/ \__/ \_
Continue reading on narkive:
Loading...