I thought someone might find this interesting - a story from a couple of
days ago:

      http://www.gamesetwatch.com/2009/07/source_code_for_over_a_dozen_7.php

Cheers,

Jeremy

On Sat, Jul 4, 2009 at 7:49 PM, Neil Bradley <nb@...> wrote:
>> and when I made the $FF opcode into data, it generated an output file
just
>> fine, and the few missing code sections all look to be there now...

> Sweet! Can you send me the latest template?
http://www.gtoal.com/apple1/template.apple1

The updated disassembly is http://www.gtoal.com/apple1/apple1.asm.txt

>> I haven't implemented the opcodes for anything more than a disassembly,
so
>> if you try to emit C code, you'll get assertions - guaranteed.
Well, first priority is just to report which opcodes are missing, and see
how much work there is left to do.

>> Anyway, here's my current sandbox:

> http://www.synthcom.com/~nb/OrionV0.28Src.zip
Thanks.

> Be forewarned it's a sandbox drop, not really well tested, and I threw
> those extra instructions in when you posted the email, so beware. ;-)

Gotcha.

G


[Non-text portions of this message have been removed]

> > Remind me how to run this... I tried recomp -template
> > templates\template.apple1 -disasm apple1.asm but there's no output,
> > just a warning about an entry vector, and an unknown opcode at E27F...
> and when I made the $FF opcode into data, it generated an output file just
> fine, and the few missing code sections all look to be there now...

Sweet! Can you send me the latest template?

> My copy of the 6502.c must be out of date, the assertion line no doesn't
> match up with the code. Though I suspect it's the final else in AST6502Misc
> that's catching it. But not enough info to tell which opcode cover is
> missing.
>
> If you could email me the most recent code drop, I can put some hours into
> it in the car while we're driving north... of all the supported
> architectures, the 6502 is the one I've done the most coding in, I shouldn't
> have too much difficulty filling in a missing opcode or two...

I haven't implemented the opcodes for anything more than a disassembly, so
if you try to emit C code, you'll get assertions - guaranteed.

Anyway, here's my current sandbox:

http://www.synthcom.com/~nb/OrionV0.28Src.zip

Be forewarned it's a sandbox drop, not really well tested, and I threw
those extra instructions in when you posted the email, so beware. ;-)

> I had forgotten quite how close to completion this thing was, and how slick
> it was... it's quite fun working on it again.

Hehehe. Well, it actually does work for Ms Pacman and Asteroids. I even
gave a demo to VMWare in 2005 on it. It was fun watching a room full of
academics argue over its value. ;-)

-->Neil

----------------------------------------------------------------------------
C. Neil Bradley - EVERYBODY KERNEL PANIC!

> That's pretty good! Comparing it manually to
> http://www.brouhaha.com/~eric/retrocomputing/apple/apple1/basic/a1basic.dis
>
> all I've seen so far is a missing label at E05D, and a couple of code
> sections - one starting at EC01 whose last instruction is a jmp at EC1B, and
> one at EC40 whose last instruction is a jmp at EC50; and the last starting
> at EE00 which goes on to join the rest of the code apart from an $FF
> instruction at EE33.
>
> then there's another missing code section from EF4E to the jmp at EFAB.
> Code starts again at EFB3.
>
> I don't see any data disassembled as code (which I do in eric's manual
> disassembly!)

Well, that's lucky! Usually it completely goes off into the weeds when it
attempts to disassemble data.

> > The recompiler portion of it won't work becaus I didn't do any of the C
> > emission for the new instructions. Enjoy.
> New instructions? Are they using some undocumented opcodes, or was the 6502
> module not complete?

The 6502 module wasn't complete and probably still isn't.

> Remind me how to run this... I tried recomp -template
> templates\template.apple1 -disasm apple1.asm but there's no output,
> just a warning about an entry vector, and an unknown opcode at E27F...

That's correct. It'll warn you about the entry vector because there isn't
one at 0xfffe or whatever the reset vector is.

The "unknown opcode" either means it's hitting an unimplemented
instruction in the disassembler, the disassembler has a bug, or it's off
in the weeds because it's trying to disassemble data.


-->Neil

----------------------------------------------------------------------------
C. Neil Bradley - EVERYBODY KERNEL PANIC!

On Sat, Jul 4, 2009 at 7:06 PM, Graham Toal <gtoal@...> wrote:
>
>  Remind me how to run this... I tried recomp -template
> templates\template.apple1 -disasm apple1.asm but there's no output,
> just a warning about an entry vector, and an unknown opcode at E27F...
>

and when I made the $FF opcode into data, it generated an output file just
fine, and the few missing code sections all look to be there now...

My copy of the 6502.c must be out of date, the assertion line no doesn't
match up with the code.  Though I suspect it's the final else in AST6502Misc
that's catching it.  But not enough info to tell which opcode cover is
missing.

If you could email me the most recent code drop, I can put some hours into
it in the car while we're driving north... of all the supported
architectures, the 6502 is the one I've done the most coding in, I shouldn't
have too much difficulty filling in a missing opcode or two...

I had forgotten quite how close to completion this thing was, and how slick
it was... it's quite fun working on it again.

G


[Non-text portions of this message have been removed]

On Sat, Jul 4, 2009 at 5:08 PM, Neil Bradley <nb@...> wrote:

>  > It's looking for the vectors at 0xfff8-0xffff, which is probably all 0s
> > unless you load something there.
>
> Well, that wouldn't have worked as it hit several opcodes that weren't
> implemented.
>
> Here is a new recomp.exe, template, and ROM image for the Apple1:
>
> http://www.synthcom.com/~nb/OrionBinaryV0.28.zip
>
> And here's what I have so far for the Apple 1 disassembly. I think it's
> pretty complete, though there are sections of "db"s that warrant closer
> analysis that I couldn't get working.
>
> http://www.synthcom.com/~nb/apple1.asm.txt
>

That's pretty good!  Comparing it manually to
http://www.brouhaha.com/~eric/retrocomputing/apple/apple1/basic/a1basic.dis
all I've seen so far is a missing label at E05D, and a couple of code
sections - one starting at EC01 whose last instruction is a jmp at EC1B, and
one at EC40 whose last instruction is a jmp at EC50; and the last starting
at EE00 which goes on to join the rest of the code apart from an $FF
instruction at EE33.

then there's another missing code section from EF4E to the jmp at EFAB.
Code starts again at EFB3.

I don't see any data disassembled as code (which I do in eric's manual
disassembly!)

cpu 6502;
entry 0xe000, 0xe00c, 0xe018, 0xe035, 0xe04b, 0xe109, 0xe121, 0xe12c,
0xe1d7, 0xe222,
       0xe8d6, 0xee3e, 0xee4c, 0xee52, 0xeee4, 0xeef6, 0xef00, 0xefec,
0xeff2, 0xe16c,
       0xe733, 0xe7a4, 0xe7b1, 0xe7c1, 0xe7e2, 0xe7f8, 0xe817, 0xe828,
0xe8a5, 0xe93a,
       0xe950, 0xe97e, 0xeea0, 0xeea6, 0xeec9, 0xeed1, 0xef10, 0xef1e,
0xef24, 0xef4e,
       0xe05d, 0xec01, 0xec40, 0xee00, 0xef4e, 0xefb3,
       apple1basic;
moduleprefix apple1basic;
coderegion 0xe000-0xefff;
ramregion 0x0000-0x1000; // Work RAM
loadimage 0xe000, "3rdparty/roms/apple1/apple1basic.bin";
inlinedata 0xec1e-0xec1f;
inlinedata 0xec20-0xec3f;
inlinedata 0xec53-0xedff;



>
> The recompiler portion of it won't work becaus I didn't do any of the C
> emission for the new instructions. Enjoy.
>

New instructions?  Are they using some undocumented opcodes, or was the 6502
module not complete?

Remind me how to run this... I tried recomp -template
templates\template.apple1 -disasm apple1.asm but there's no output,
just a warning about an entry vector, and an unknown opcode at E27F...

btw I'm on vacation for 2 weeks as of today, you can tell, right? :-)

thanks,

G


[Non-text portions of this message have been removed]

> It's looking for the vectors at 0xfff8-0xffff, which is probably all 0s
> unless you load something there.

Well, that wouldn't have worked as it hit several opcodes that weren't
implemented.

Here is a new recomp.exe, template, and ROM image for the Apple1:

http://www.synthcom.com/~nb/OrionBinaryV0.28.zip

And here's what I have so far for the Apple 1 disassembly. I think it's
pretty complete, though there are sections of "db"s that warrant closer
analysis that I couldn't get working.

http://www.synthcom.com/~nb/apple1.asm.txt

The recompiler portion of it won't work becaus I didn't do any of the C
emission for the new instructions. Enjoy.

-->Neil

----------------------------------------------------------------------------
C. Neil Bradley - EVERYBODY KERNEL PANIC!

> > Does someone have a copy of the Apple 1 EPROM? I'd be willing to run it
> > through Orion to see what comes out the other end.
> Thanks! I did try it myself but was stymied by some basics like it was
> assuming
> an entry point at 0000 and i couldn't see how to override that.

It's looking for the vectors at 0xfff8-0xffff, which is probably all 0s
unless you load something there.

Try adding:

loadimage 0xf000, "3rdparty\\roms\\apple1basic.bin";

To the script. That should take care of it. I'll give it a shot here and
see how it goes.

-->Neil

----------------------------------------------------------------------------
C. Neil Bradley - EVERYBODY KERNEL PANIC!

On Sat, Jul 4, 2009 at 1:58 PM, Neil Bradley <nb@...> wrote:
>> Not that the translated code looks all that great yet, but using an
>> optimising compiler backend is exactly the way to go.

>Are you referring to AST + target or just target? I would think the AST
>route is the way to go.

Of course!  LLVM is a compiler back-end - you feed it your AST and it does
all the optimisations on the AST for you, and has a choice of target code
generators.
It would actually be a good fit with Orion, although I'm not suggesting
going that
route (at least not unless it becomes more obvious how to output more
approachable C)

>> Apple 1 basic is a 4K file. He hasn't posted the source of the translated
>> version, though a lot of people are asking for it. I thought it might be
fun
>> to push it through Orion and see what it looks like. There are a couple
of
>> partial disassemblies of apple1 basic out there already (although
reportedly
>> not 100% accurate copies of the code) which will give most of the labels
and
>> jumps.

> Does someone have a copy of the Apple 1 EPROM? I'd be willing to run it
> through Orion to see what comes out the other end.

Thanks!  I did try it myself but was stymied by some basics like it was
assuming
an entry point at 0000 and i couldn't see how to override that.

I copied the bin to www.gtoal.com/apple1/ for you.

This is my half-assed attempt at starting a template.

/* Template for Apple I basic (4K)
See...
    http://www.pagetable.com/?p=32
    http://www.brouhaha.com/~eric/retrocomputing/apple/apple1/basic/
    http://home.comcast.net/~mjmahon/AP1BASIC.txt
    http://gloonk.com/programming/images/wozbasicvars.txt
  */
cpu                             6502;
moduleprefix            apple1basic;
coderegion                      0xe000-0xefff;
ramregion                       0x0000-0x1000;  // Work RAM
loadimage                       0xe000, "3rdparty\\roms\\apple1basic.bin";
entry                           0xe000, apple1basic;
inlinedata                      0xec1e-0xec1f;
inlinedata                      0xec20-0xec3f;
inlinedata                      0xec53-0xedff;



>  .
>
>
>


[Non-text portions of this message have been removed]

> Not that the translated code looks all that great yet, but using an
> optimising compiler backend is exactly the way to go.

Are you referring to AST + target or just target? I would think the AST
route is the way to go.

> Apple 1 basic is a 4K file. He hasn't posted the source of the translated
> version, though a lot of people are asking for it. I thought it might be fun
> to push it through Orion and see what it looks like. There are a couple of
> partial disassemblies of apple1 basic out there already (although reportedly
> not 100% accurate copies of the code) which will give most of the labels and
> jumps.

Does someone have a copy of the Apple 1 EPROM? I'd be willing to run it
through Orion to see what comes out the other end.

-->Neil

----------------------------------------------------------------------------
C. Neil Bradley - EVERYBODY KERNEL PANIC!

> Not that the translated code looks all that great yet,
> but using an optimising compiler backend is exactly
> the way to go. You can tell from the comments that he
> is handling a proper data flow graph which should
> allow for global optimisation (the clue is the
> references to 'phi nodes', which won't mean much
> unless you've done some advanced compiler reading)

As far as I understand it, SSA is part of LLVM so using
LLVM provides you with SSA optimised code for free,
which is good.

On the other hand, I prefer a translation that I can
easily relate to the original code. Of course flags
removal is a must to make the translated code
readable. Then the host compiler can do whatever
kind of optimisations it can. I have a project in
mind to use QEMU for that (of course that'd be for
recent processors).

> I thought it might be fun
> to push it through Orion and see what it looks like.
> There are a couple of partial disassemblies of apple1
> basic out there already (although reportedly not 100%
> accurate copies of the code) which will give most of
> the labels and jumps.

Yeah might be fun :-) I have lost all links to Orion
download (I changed my machine a few months ago and was
too lazy to backup files...).


Laurent

Michael Steil, the guy who did the Pear dynarec for Macs and who discussed the
recompiler howto in his document, has written a recompiler for Apple I and
Commodore.  I discovered this when reading about the restoration of Apple I
basic ( http://www.pagetable.com/?p=35 ) which led to a similar article about
Commodore basic.  Steil hasn't posted the Apple basic translation online but he
does have the Commodore one online at Sourceforge
(http://cbmbasic.sourceforge.net/ ) which you can fetch and look at with svn.

He hasn't posted the recompiler, but looking at the translated code it would
appear that he is using LLVM ( http://llvm.org/ ) as a back end.

Not that the translated code looks all that great yet, but using an optimising
compiler backend is exactly the way to go. You can tell from the comments that
he is handling a proper data flow graph which should allow for global
optimisation (the clue is the references to 'phi nodes', which won't mean much
unless you've done some advanced compiler reading)

Apple 1 basic is a 4K file.  He hasn't posted the source of the translated
version, though a lot of people are asking for it.  I thought it might be fun to
push it through Orion and see what it looks like.  There are a couple of partial
disassemblies of apple1 basic out there already (although reportedly not 100%
accurate copies of the code) which will give most of the labels and jumps.

G

> On Sun, Aug 31, 2008 at 2:28 AM, Neil Bradley <nb@...> wrote:
>> Yeah, I know, not strictly on topic, but I just got my expression
>> evaluator emitting correct uOps for a BASIC interpreter.
> Hi Neil - Cris Cranford over in "compilers101" is in the early stages
> of writing a BASIC as well - I can't remember if I ever dragged you
> into that group before, but if not, why not come over and join us.
> This is exactly the sort of stuff we do!  I'm actually working on a
> compiler myself at the moment too - it's about the same level as
> BASIC, and I'm playing around with different back-ends - so far I have
> an interpreter, and a C emitter, and a 3-address emitter, but I'm
> looking to do x86 code next - maybe you can help me with that..?  I
> have one 'real life' test program written for this cut-down language -
> it's a fairly powerful programmable text editor.

That would be excellent! Where/how do I get subscribed?

-->Neil

----------------------------------------------------------------------------
C. Neil Bradley - KE7IXP - The one eyed man in the land of the blind is not
                             king. He's a prisoner.

On Sun, Aug 31, 2008 at 2:28 AM, Neil Bradley <nb@...> wrote:
> Yeah, I know, not strictly on topic, but I just got my expression
> evaluator emitting correct uOps for a BASIC interpreter.

Hi Neil - Cris Cranford over in "compilers101" is in the early stages
of writing a BASIC as well - I can't remember if I ever dragged you
into that group before, but if not, why not come over and join us.
This is exactly the sort of stuff we do!  I'm actually working on a
compiler myself at the moment too - it's about the same level as
BASIC, and I'm playing around with different back-ends - so far I have
an interpreter, and a C emitter, and a 3-address emitter, but I'm
looking to do x86 code next - maybe you can help me with that..?  I
have one 'real life' test program written for this cut-down language -
it's a fairly powerful programmable text editor.


Graham

Yeah, I know, not strictly on topic, but I just got my expression
evaluator emitting correct uOps for a BASIC interpreter.

Anyway, a sequence like this:

Dim i(10, 20, 30) As Double
Dim j As Int
j=5
i(0,0,0)=i(1,2,3)+i(4,5,6)

Generates:

1    : EXEC_VAR_CREATE     : Local [0] of type EVAR_SIGNED_INT
1    : EXEC_ASSIGN         : Local [0](10) = 10
1    : EXEC_VAR_CREATE     : Local [1] of type EVAR_SIGNED_INT
1    : EXEC_ASSIGN         : Local [1](20) = 20
1    : EXEC_VAR_CREATE     : Local [2] of type EVAR_SIGNED_INT
1    : EXEC_ASSIGN         : Local [2](30) = 30
1    : EXEC_VAR_CREATE     : Global [0] "i" of type EVAR_DOUBLE 3 dimensions
1    : EXEC_VAR_DELETE     : Local [0]
1    : EXEC_VAR_DELETE     : Local [1]
1    : EXEC_VAR_DELETE     : Local [2]
2    : EXEC_VAR_CREATE     : Global [1] "j" of type EVAR_SIGNED_INT
3    : EXEC_ASSIGN         : Global [1](5) = 5
4    : EXEC_VAR_CREATE     : Local [0] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [0](1) = 1
4    : EXEC_VAR_CREATE     : Local [1] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [1](2) = 2
4    : EXEC_VAR_CREATE     : Local [2] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [2](3) = 3
4    : EXEC_VAR_CREATE     : Local [3] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [3](4) = 4
4    : EXEC_VAR_CREATE     : Local [4] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [4](5) = 5
4    : EXEC_VAR_CREATE     : Local [5] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [5](6) = 6
4    : EXEC_OP             : Global [0](0.000000) "i" = Global [0](0.000000)
"i" + Global [0](0.000000) "i"
4    : EXEC_VAR_DELETE     : Local [0]
4    : EXEC_VAR_DELETE     : Local [1]
4    : EXEC_VAR_DELETE     : Local [2]
4    : EXEC_VAR_DELETE
       : Local [3]
4    : EXEC_VAR_DELETE     : Local [4]
4    : EXEC_VAR_DELETE     : Local [5]

Now, let's go for the nasty (believe it or not, the sequence of uOps it
generates is 100% correct):

Dim i(10, 20, 30) As Double
Dim j As Int
j=5
i(0,0,0)=i(1, i(7,8,9) * i(9,10,11) /i( i(5,2,1), 14*7/9, i(7,3,2)),14)+i(4*j +
7,5,6)

Generates:

1    : EXEC_VAR_CREATE     : Local [0] of type EVAR_SIGNED_INT
1    : EXEC_ASSIGN         : Local [0](10) = 10
1    : EXEC_VAR_CREATE     : Local [1] of type EVAR_SIGNED_INT
1    : EXEC_ASSIGN         : Local [1](20) = 20
1    : EXEC_VAR_CREATE     : Local [2] of type EVAR_SIGNED_INT
1    : EXEC_ASSIGN         : Local [2](30) = 30
1    : EXEC_VAR_CREATE     : Global [0] "i" of type EVAR_DOUBLE 3 dimensions
1    : EXEC_VAR_DELETE     : Local [0]
1    : EXEC_VAR_DELETE     : Local [1]
1    : EXEC_VAR_DELETE     : Local [2]
2    : EXEC_VAR_CREATE     : Global [1] "j" of type EVAR_SIGNED_INT
3    : EXEC_ASSIGN         : Global [1](5) = 5
4    : EXEC_VAR_CREATE     : Local [0] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [0](1) = 1
4    : EXEC_VAR_CREATE     : Local [1] of type EVAR_SIGNED_INT
4    : EXEC_VAR_CREATE     : Local [2] of type EVAR_UNSPECIFIED
4    : EXEC_VAR_CREATE     : Local [3] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [3](7) = 7
4    : EXEC_VAR_CREATE     : Local [4] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [4](8) = 8
4    : EXEC_VAR_CREATE     : Local [5] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [5](9) = 9
4    : EXEC_VAR_CREATE     : Local [6] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [6](9) = 9
4    : EXEC_VAR_CREATE     : Local [7] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [7](10) = 10
4    : EXEC_VAR_CREATE     : Local [8] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [8](11) = 11
4    : EXEC_OP             : Local [2](0.000000) = Global [0](0.000000) "i" *
Global [0](0.000000) "i"
4    : EXEC_VAR_DELETE     : Local [3]
4    : EXEC_VAR_DELETE     : Local [4]
4    : EXEC_VAR_DELETE     : Local [5]
4    : EXEC_VAR_DELETE     : Local [6]
4    : EXEC_VAR_DELETE     : Local [7]
4    : EXEC_VAR_DELETE     : Local [8]
4    : EXEC_VAR_CREATE     : Local [3] of type EVAR_SIGNED_INT
4    : EXEC_VAR_CREATE     : Local [4] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [4](5) = 5
4    : EXEC_VAR_CREATE     : Local [5] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [5](2) = 2
4    : EXEC_VAR_CREATE     : Local [6] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [6](1) = 1
4    : EXEC_ASSIGN         : Local [3](0) = Global [0](0.000000) "i"
4    : EXEC_VAR_DELETE     : Local [4]
4    : EXEC_VAR_DELETE     : Local [5]
4    : EXEC_VAR_DELETE     : Local [6]
4    : EXEC_VAR_CREATE     : Local [4] of type EVAR_SIGNED_INT
4    : EXEC_VAR_CREATE     : Local [5] of type EVAR_UNSPECIFIED
4    : EXEC_OP             : Local [5](98) = 14 * 7
4    : EXEC_OP             : Local [4](10) = Local [5](98) / (FP) 9
4    : EXEC_VAR_DELETE     : Local [5]
4    : EXEC_VAR_CREATE     : Local [5] of type EVAR_SIGNED_INT
4    : EXEC_VAR_CREATE     : Local [6] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [6](7) = 7
4    : EXEC_VAR_CREATE     : Local [7] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [7](3) = 3
4    : EXEC_VAR_CREATE     : Local [8] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [8](2) = 2
4    : EXEC_ASSIGN         : Local [5](0) = Global [0](0.000000) "i"
4    : EXEC_VAR_DELETE     : Local [6]
4    : EXEC_VAR_DELETE     : Local [7]
4    : EXEC_VAR_DELETE     : Local [8]
4    : EXEC_OP             : Local [1](0) = Local [2](0.000000) / (FP) Global
[0](0.000000) "i"
4    : EXEC_VAR_DELETE     : Local [3]
4    : EXEC_VAR_DELETE     : Local [4]
4    : EXEC_VAR_DELETE     : Local [5]
4    : EXEC_VAR_DELETE     : Local [2]
4    : EXEC_VAR_CREATE     : Local [2] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [2](14) = 14
4    : EXEC_VAR_CREATE     : Local [3] of type EVAR_SIGNED_INT
4    : EXEC_VAR_CREATE     : Local [4] of type EVAR_UNSPECIFIED
4    : EXEC_OP             : Local [4](20) = 4 * Global [1](5)
4    : EXEC_OP             : Local [3](27) = Local [4](20) + 7
4    : EXEC_VAR_DELETE     : Local [4]
4    : EXEC_VAR_CREATE     : Local [4] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [4](5) = 5
4    : EXEC_VAR_CREATE     : Local [5] of type EVAR_SIGNED_INT
4    : EXEC_ASSIGN         : Local [5](6) = 6
4    : EXEC_OP             : Global [0](0.000000) "i" = Global [0](0.000000)
"i" + Global [0](0.000000) "i"
4    : EXEC_VAR_DELETE     : Local [0]
4    : EXEC_VAR_DELETE     : Local [1]
4    : EXEC_VAR_DELETE     : Local [2]
4    : EXEC_VAR_DELETE     : Local [3]
4    : EXEC_VAR_DELETE     : Local [4]
4    : EXEC_VAR_DELETE     : Local [5]

So yes, folks, you too can write lousy code and we'll still run it. ;-)

The final values being pulled from (and written to) the arrays aren't
working yet, but they will soon...

-->Neil

----------------------------------------------------------------------------
C. Neil Bradley - KE7IXP - The one eyed man in the land of the blind is not
                             king. He's a prisoner.

Steve Coleman wrote:
>
> You could accomplish that with a the use of an OS hypervisor that traps
> memory writes via a hardware protection access and journals that to disk
> for future rewind/replay. Doing it at the application/compiler level
> would be difficult because there would be no synchronization between
> applications on the system thus the replay would happen out of sequence
> and the results would possibly give you a different outcome. The results
> of going forward again would be non-deterministic, though possibly still
> good for Monte Carlo analysis.
>
> I do recall somebody has done something like this at the OS level for
> the purpose of working with malware, but at the moment I can't remember
> who that was. I guess 'I' need to go rewind my grey matter a little, or
> go back and re-search all my bookmarks. ;)
>
> Also, a related topic where you can learn some of the tricks of the
> trade happen in the fault tolerance 'process migration' arena since
> during the migration of a process all memory writes need to be
> journalled for replication across both systems. Its all interesting stuff.
>
> btw - Looks like your/somebody's system is already rewound back to
> '2008-02-24' because that is the time stamp I got on the email you
> posted and I had to scroll back in my inbox just to find that one unread
> message.
>
> Jeremy Smith jeremy-at-xanadu.net |yahoogroups/fwdHome| wrote:
> > Hi all,
> >
> > In 2006, or thereabouts, I had an idea for a computer which you could
> > 'rewind' and take it back to any earlier instance. It would do this by
> > logging every memory write, and every hardware write.
> >
> > Problems are that you couldn't run a network client like this, because
> > the state of the network would change in the interim. However, you
> could
> > have every computer on the network also rewindable.
> >
>
>
Hi,

Thanks for the ideas.

The reason for the timestamp is I hit 'Send All Messages' by mistake and
it posted that old message. Sorry about that!

Jeremy.

You could accomplish that with a the use of an OS hypervisor that traps
memory writes via a hardware protection access and journals that to disk
for future rewind/replay. Doing it at the application/compiler level
would be difficult because there would be no synchronization between
applications on the system thus the replay would happen out of sequence
and the results would possibly give you a different outcome. The results
of going forward again would be non-deterministic, though possibly still
good for Monte Carlo analysis.

I do recall somebody has done something like this at the OS level for
the purpose of working with malware, but at the moment I can't remember
who that was.  I guess 'I' need to go rewind my grey matter a little, or
go back and re-search all my bookmarks. ;)

Also, a related topic where you can learn some of the tricks of the
trade happen in the fault tolerance 'process migration' arena since
during the migration of a process all memory writes need to be
journalled for replication across both systems. Its all interesting stuff.

btw - Looks like your/somebody's system is already rewound back to
'2008-02-24' because that is the time stamp I got on the email you
posted and I had to scroll back in my inbox just to find that one unread
message.

Jeremy Smith jeremy-at-xanadu.net |yahoogroups/fwdHome| wrote:
> Hi all,
>
> In 2006, or thereabouts, I had an idea for a computer which you could
> 'rewind' and take it back to any earlier instance. It would do this by
> logging every memory write, and every hardware write.
>
> Problems are that you couldn't run a network client like this, because
> the state of the network would change in the interim. However, you could
> have every computer on the network also rewindable.
>

Hi all,

In 2006, or thereabouts, I had an idea for a computer which you could
'rewind' and take it back to any earlier instance. It would do this by
logging every memory write, and every hardware write.

Problems are that you couldn't run a network client like this, because
the state of the network would change in the interim. However, you could
have every computer on the network also rewindable.

The problem i

Jeremy.

http://video.aol.com/video-detail/zx-spectrum-emulator-on-atari-8bit/1839861807

Im not sure if this is true static binary recompiler (I haven't
searched for additional information, as most I could find was in polish).

I have been playing around with a simillar fantasy for some time
(recompiling code for instance an vintage arcade machine, so it can be
run on a vintage computer system)

-jkm

On Tue, Mar 25, 2008 at 6:24 AM, jankaspermartinsen
<jankaspermartinsen@...> wrote:
>
> --- In staticrecompilers@yahoogroups.com, "Graham Toal" <gtoal@...> wrote:
> >
> > I had a thought today (while watching "The King of Kong" as it
> > happens) which is this...
> >
> > we can emulate classic video games at what - 1000 times faster than
> > real time? 10,000 times?
> >
> > So wouldn't it be possible to set up a real-time emulation that worked
> > out the best play
> > for any game by forking at each decision point and playing ahead -
> > either a fixed window like
> > 10 seconds, or maybe to the end of the screen - so that it could work
> > out a perfect game
> > in real time...  much the same as classic AI techniques are used to
> > lookahead in other
> > games like checkers, backgammon, scrabble, whatever...
> >
> > I'm wondering just how much speed is needed to pull this off, and what
> > pruning algorithms.
> >
> > Can you for example play a perfect game just by knowing that you can't
> > die in the next
> > 10 seconds, for example, or do some games require more long-term
> > strategy & planning?
> >
> > I'm thinking of the classic games like mspac, donkey kong, qbert etc?
> >
> > I know that various people have worked on AI players for games like
> > pacman, but I believe
> > they're just heuristics, I don't think anyone has thrown power &
> > classic AI techniques
> > at it yet?
> >
> > Would be a truly cool student project by the way, if we have any
> > students reading...
> >
> > G
> >
>
> Hi,
>
> I've been thinking about that too, maybe making some sort of game
> tree. But if I remeber correctly, I feared that the game tree would
> easily be very large (but please prove me wrong)
>
> jkm

well, think about a game like pacman - there are a very few *right*
choices to make and a lot of *wrong* choices.  So although the
branching ratio is indeed large, I have a strong suspicion that
fruitless paths (no pun intended) will die off early.  It may be
sufficient to only ever guarantee that you can live for the next 10
seconds for example, to be able to always win, rather than look ahead
to the end of the screen.

G
PS Your yahoo email address is bouncing (acct disabled) so I posted my
reply to the group instead.

Graham Toal wrote:
>
> I had a thought today (while watching "The King of Kong" as it
> happens) which is this...
>
> we can emulate classic video games at what - 1000 times faster than
> real time? 10,000 times?
>
> So wouldn't it be possible to set up a real-time emulation that worked
> out the best play
> for any game by forking at each decision point and playing ahead -
> either a fixed window like
> 10 seconds, or maybe to the end of the screen - so that it could work
> out a perfect game
> in real time... much the same as classic AI techniques are used to
> lookahead in other
> games like checkers, backgammon, scrabble, whatever...
>
> I'm wondering just how much speed is needed to pull this off, and what
> pruning algorithms.
>
> Can you for example play a perfect game just by knowing that you can't
> die in the next
> 10 seconds, for example, or do some games require more long-term
> strategy & planning?
>
> I'm thinking of the classic games like mspac, donkey kong, qbert etc?
>
> I know that various people have worked on AI players for games like
> pacman, but I believe
> they're just heuristics, I don't think anyone has thrown power &
> classic AI techniques
> at it yet?
>
> Would be a truly cool student project by the way, if we have any
> students reading...
>
> G
>
>

I had a vague idea of the same kind of thing, used to find the end
sequence of a game, but I had the problem of doing it statically - it
would not be able to, say, follow branches which were self-modified.

However, because this is done dynamically via emulation, it should work quite
well.

Cheers,

Jeremy.

I had a thought today (while watching "The King of Kong" as it
happens) which is this...

we can emulate classic video games at what - 1000 times faster than
real time? 10,000 times?

So wouldn't it be possible to set up a real-time emulation that worked
out the best play
for any game by forking at each decision point and playing ahead -
either a fixed window like
10 seconds, or maybe to the end of the screen - so that it could work
out a perfect game
in real time...  much the same as classic AI techniques are used to
lookahead in other
games like checkers, backgammon, scrabble, whatever...

I'm wondering just how much speed is needed to pull this off, and what
pruning algorithms.

Can you for example play a perfect game just by knowing that you can't
die in the next
10 seconds, for example, or do some games require more long-term
strategy & planning?

I'm thinking of the classic games like mspac, donkey kong, qbert etc?

I know that various people have worked on AI players for games like
pacman, but I believe
they're just heuristics, I don't think anyone has thrown power &
classic AI techniques
at it yet?

Would be a truly cool student project by the way, if we have any
students reading...

G

It's been a few years since I looked at UQBT and I'm sure when I last
looked at it, most of the papers associated with it were on commercial
servers and there was no source available.  Well, at some point that
must have changed, because there's a 300+ page document online here:

http://www.experimentalstuff.com/Technologies/uqbt/uqbt.pdf

and the sources are here:

http://www.experimentalstuff.com/Technologies/uqbt/index.html

Has to be worth a careful read (esp for the Orion team)...

Also the related 'walkabout' project is at that same site:
http://www.experimentalstuff.com/Technologies/Walkabout/index.html


G

On 10/25/07, Ed <emandy@...> wrote:
> I'd say at this point I'm just gathering information. RACE! is not
> currently my main project, but my main project (http://
> cfc2english.blogspot.com/) ties into the desire to have a good,
> portable, NGPC emulator.
>
> Anyway, I was pointed toward this group by a friend, so I figured I'd
> join and soak up some knowledge.

You're very welcome, and as a fellow GP32/GP2X (and arcade game)
enthusiast, it'll be good to have someone who is interested in the
same areas to bounce ideas around with.

First big point to clarify - you won't be able to write a general
purpose emulator using any of the SBT techniques we can suggest.  What
you will be able to do is take your existing emulator and a specific
game or two, and generate binaries corresponding to those specific
games.  If your host platform (the NGPC which I'm not familiar with)
is not just bare metal, you might also be able to translate the
operating system roms to native code as well.

The project you pointed to in your blog - one specific game, and you
want to tweak some of the strings in that game - on the other hand is
a perfect application for static translation.  You've probably
disassembled most of it already and have a good idea of the control
flow.

But if you really need a faster 'traditional' emulator that can run
arbitrary binaries, then you should look at dynamic binary translation
(also known as dynamic recompiling), which to some extent is a
different set of solutions.  However in my opinion the best emulation
will come from a combination of dynamic and static translation, in
particular from doing an initial pre-translation at startup time, with
a fall-back dynarec for code which could not be translated before
startup.  Also some sort of long-term memory for games to feed back
things like code addresses and jump targets can migrate code over time
from primarily dynamic to primarily pre-compiled.  The big win from
this is that the more you can pre-compile, the better optimisations
you can apply.

If you haven't already seen it, there's a "Howto" document you might
find worth a read just to get an initial introduction to the subject
(www.gtoal.com/sbt); and there's also a fairly well developed (but not
completed) project on the go to produce a multi-target translator.
However before getting into that project (mostly written by Neil
Bradley; it's called Orion) I woud recommend that you have a go at it
yourself using the Howto and your existing emulator.  Once you've got
a basic translation working using code you're familiar with, you might
then want to consider working with us on Orion in order to consolidate
effort.  Orion attempts to perform several optimizations in a portable
way which you would be unlikely to put in your own code first time
round.

However even relatively unoptimised translated code can be *much*
faster than emulated code - for a specific GP32 example (and this is
*not* heavily optimised, it was my first ever attempt at a translation
while I was still teaching myself the basics) have a look at
Tailgunner for the gp32: http://www.gtoal.com/athome/tailgunner/gp32/

(By the way, the strange breaking of the switch statement into several
files was not a GP32 thing; it came from having attempted a port to a
68000 platform - PalmOS - which has a 32K limit on procedures.  And
no, the Palm wasn't fast enough, at least at the time.  Probably
current ones would be.)

regards

Graham

Hi guys.  I just joined the group and thought I should say hi and
introduce myself.  I'm mostly interested in handheld game console
development (GB, NGPC, GBA, GP32, GP2X, NDS, etc.).

One of my projects is a NGPC (Neo Geo Pocket Color) emulator called
"RACE!", and I would like it to run on the NDS (Nintendo DS).
Currently, though, it only runs at about 1/3 of full speed.  It does
run at full speed on the GP2X, and (with some tricks) GP32.  I have
pretty much convinced myself that the emulation of the TLCS900h
(NGPC's main CPU) is the main issue.  I have found some ways to
further optimize it for the ARM CPUs (that I have yet to implement),
but I'm curious about other emulation and recompilation techniques.

I'd say at this point I'm just gathering information.  RACE! is not
currently my main project, but my main project (http://
cfc2english.blogspot.com/) ties into the desire to have a good,
portable, NGPC emulator.

Anyway, I was pointed toward this group by a friend, so I figured I'd
join and soak up some knowledge.

The latest entry into the decompiler arena that I know of is HexRays.

http://www.hexblog.com/hexrays/manual/
http://www.hexblog.com/

Its the one project to keep an eye on if you are into reverse
engineering. The same people that made IDA Pro so great for reversing
binaries are now taking the technology to the next level. Its still a
work in process but it is being actively worked on by people that
really know what they are doing.


--- In staticrecompilers@yahoogroups.com, "Graham Toal" <gtoal@...> wrote:
>
> I found this on the net today.  It's a decompiler from X86 to C, in
> some ways similar to the various SBTs that we've all written.  However
> where it's different is that after the initial - very basic -
> translation, there's an interactive tool for building up the simple
> code into a higher-level version, renaming variables, etc.
>

I found this on the net today.  It's a decompiler from X86 to C, in
some ways similar to the various SBTs that we've all written.  However
where it's different is that after the initial - very basic -
translation, there's an interactive tool for building up the simple
code into a higher-level version, renaming variables, etc.

Watch this screencast...

   http://idc.sourceforge.net/viva.avi

which comes from here:
   http://idc.sourceforge.net/wiki/

also some screenshots:
   http://idc.sourceforge.net/wiki/Screenshots.html

I have to say that I have some doubts about this - I'm pretty sure
that it could do a lot more automatic reconstruction of sequential
statements into expressions ... but it's interesting nevertheless if
only for the ingenuity of the interactive editor.

G

> Here is an idea how to possibly solve this problem. playing through
> phoenix time after time is exhausting for one person, but if 1000
> people played it, each doing their part? Here is what to do; Write an
> emulator-translator (the user will only see this as an emulator),
> this emulator-translator should send the translated code to some
> central server that will keep track of all the translated code. Based
> on code on this server, we can translate the game to a smaller,
> handheld device, without doing all the playing ourselves. As a bonus,
> the finished translated game could be sent to all the players, as
> some sort of reward.
>
> What do you think about the idea?

I think you may be on to something, but maybe the details aren't
quite optimum yet.  I'm sorry I haven't posted sooner, and indeed
I'm so busy at the moment I can't write a proper mail just now either;
but I will follow up later this week with some ideas.

Quick comment: did you rely 100% on finding the jumps by playtesting
or did you also do a tree-walk of the code with a disassembler?

If you also want to do manually disassembly you can find things like
jump tables which playtesting and auto-disassembly might miss.

Anyway, what I'll post about later is on what information exactly
you want to store and/or report, and on the transparency of
fallback emulation.


Graham

Years ago, I wrote a phoenix emulator. After some playing, it quickly
was placed together with old projects on my harddrive. Then some
months ago, I read the article by Graham Toal about static binary
translation. I found the phoenix emulator again, changed it abit, so
it translated phoenix into j2me code. There was several issues using
Grahams approach to port old arcade games, for instance there were
smart tricks to improve speed and size of the translated game (which
were needed since my cellphone was quite slow and had little memory).
However, the biggest problem for me was that I had to play phoenix
quite alot in order to translate all the code, infact I played it so
many times that I had lost interest before I uploaded the finished
translated game to my cellphone, and this was a 16kb game (if I
remember correctly). My initial idea was to use the technique to port
games from the early '90s, however some of the ones I wanted to port
was larger than 16kb, and based on the  experiences from the phoenix
translation I felt that it was a very bad idea.

Here is an idea how to possibly solve this problem. playing through
phoenix time after time is exhausting for one person, but if 1000
people played it, each doing their part? Here is what to do; Write an
emulator-translator (the user will only see this as an emulator),
this emulator-translator should send the translated code to some
central server that will keep track of all the translated code. Based
on code on this server, we can translate the game to a smaller,
handheld device, without doing all the playing ourselves. As a bonus,
the finished translated game could be sent to all the players, as
some sort of reward.

What do you think about the idea?

jkm

You might be interested, I made a static compiled version of the
phoenix arcade game (the only emulator I have ever written). It can be
run on j2me enabled devices, which is compatible with nokia s60. It
still slow (on old devices), but I belive this has something to do
with graphic routines on the phone(but im not sure). More details, see:

http://kaspermartinsen.googlepages.com/sbt

-jkm