64DD Test Homebrews

IIRC, even on the IPL, I don't think it wrote the SET_DTYPE command?

So yep, it's probably not necessary for most stuff, and the important thing is to grab the disk type first so it can calculate everything else.

Yep, I think copying full blocks is a case of setting the Xferblks bit, then checking for BM_INT in a loop until all the sectors for that block are transferred.
It would be better to alloc the memory for a full block as well, then we can be confident of grabbing the data fast enough and that it should be valid data.

OzOnE

 

I would make readDiskBlock/LBA() for that and not just use readDiskSector/LBA().
I'll deal with readDiskSector to support all Disk Types just right and then I'll try to do readDiskBlock. The next thing would be to make a Memory Viewer at this point to make sure that everything is right.

Better to make another homebrew for that.

EDIT: 64DD Sector Viewer is done! However it doesn't recognize the Disk Type yet and will probably not read RAM Data yet.
It's tested and reads LBAs just fine.
It's called 64ddview.
Source Code still at: https://github.com/LuigiBlood/64dd
ROM is there: https://raw.githubusercontent.com/LuigiBlood/64dd/master/64ddview/64ddview.v64

EDIT2:
There's a hole in the Sector reading, always at 0x20-0x7F, it's always 00s. That's not normal. I may know how to fix it though.
One other thing, I'll leave this log with just me in it:
[01:51] <@Seru-kun> Just do some trickery with the last two bits of LBA
[01:52] <@Seru-kun> LBA 0: %00, LBA 1: %01, LBA 2: %10, LBA 3: %11
[01:52] <@Seru-kun> 0 XOR 0 = 0, 0 XOR 1 = 1, 1 XOR 0 = 1, 1 XOR 1 = 0
[01:52] <@Seru-kun> When it's 0, it's Block 0
[01:52] <@Seru-kun> When it's 1, it's Block 1
[01:53] <@Seru-kun> Easy.

LBAs are weird here, but it's accurate to the SDK.

 

Well done on the Viewer - that was quick.

Working on mine as well.

Are you saying the odd and even blocks are really encoded like that? That's really strange.
That must be a bit of obfuscation from the point of file of the file system?

I'm on IRC atm btw (#n64dev).
I'll be trying to log on a bit more often.

LuigiBlood - do you have a 64DD btw, I can't remember?
It's just that Jimmy says he's testing the code on his, but I thought you had one too?

OzOnE.

 

Apparently, the SDK works like that for the LBAs, jrra told me it's because to copy several blocks it's actually faster like that since the disk is still turning. It's not an obfuscation but rather something that makes it faster for sequential blocks copy. Makes sense there.

And damn I just missed you on IRC.
Also, Seru-kun is me. It's an alternate nick I use on IRC because I don't like the char limit (I appear as "LuigiBloo" because of that and I don't like that).

And no, I don't have a 64DD. I've been doing this blindly, tested a few others who were willing to test for me, even though it's pretty slow.
I've said it on the first post of this topic.

 

I've been lurking here for a while and I love the progress you guys are making! If I had a 64DD I'd send you one immediately to use for this, I missed out on that chance long ago though.
It's all programmed in C right? I want to help in some way, but I only know absolute basics(if it's C with libdragon I'll learn GB programming first then move up to N64).
Can't wait to see what progress comes from you guys in the future, I'd really love a DD emulator!

 

So I might have to write up a post, giving some updates about what's going on. Sorry for not giving much updates, but on the development itself, I'll give you a sum up on why it's taking some time, and some other things I feel like talking about right now:

First, 64ddview, the Disk Sector Viewer, is having trouble on reading the disk data. Timing problems are really annoying, and I'm mostly trying to bypass the timing so that once the data is ready to be read, it reads it immediately. The thing is, we're doing the right thing to read a specific sector. However, when it's time to read it, something is wrong. Either there's a hole of 00s in the middle, sometimes moved a bit as well, or we don't even get data, which is worse. However, the data we managed to get does correspond with the existing dump of F-Zero X Expansion Kit, so one thing for sure, the disks aren't dead yet, so there's no time to lose now.

Second, I have to apologize about something. I remember saying a few years ago that current disk dumps cannot be emulated because they're incomplete. That's actually not true. By actually doing some smart stuff we can actually get something to work, and fill what's missing, or so jrra says. However that does not the change the fact that current dumps are incomplete.

What baffles me the most is that I'm sure that stuff were dumped just fine way back, but not on the web. The fact that it took 15 years to actually get things going: So, about preservation, I only see it happening right now. Sorry but that's how I feel.

Third, unlike the people behind what I've kind of talked about just above, I try to give as much information as possible, thus I'm currently writing 64DD docs on my GitHub.
You can look at its current state right now: https://github.com/LuigiBlood/64dd/wiki
I'm still crediting the people who still gave a few docs here and there, and this pretty much sums it all up in one single place.

One more thing, if there's something you don't actually understand, PM me, talk to me on Twitter, on IRC, places I am on, and I'll gladly try to explain; same deal goes if you actually have more information, or if something is really not clear on the 64DD wiki and needs to be rewritten, I will still be listening.

Just because I don't talk much doesn't mean that I'm dead and disappeared off the web, that never really happened to me. I'm ALWAYS on.

 

Hi, LB,

I know what you mean about the preservation thing - it's a shame that the disk images weren't made available a long time ago by whoever did the first few rips.

I imagine it is possible to recreate the system blocks for the few publicly released images too, but getting a decent full dump is still the "easiest" way to go.
We can then modify the boot blocks from one disk to point to the correct code block(s) for Dezaemon etc.

A fair bit of info on the MFS and boot stuff is given in the N64 SDK as well.

Thanks for the mention on the Wiki btw. Some great info on there already.

I would pop on IRC more often, but now I have "Man Flu" (tm), so I'm only on the PC in short bursts atm before I cough and sneeze over the monitor and whole room. lol

OzOnE.

 

Let me give you some updates. Merry Christmas!

First: I'm now using N64 SDK, and the homebrew will most likely be based on an existing exemple inside the SDK called "pfs".
My first test was to make a RTC Test homebrew, unfortunately, it doesn't work on MESS for some reason.
For now, it's untested on real HW, and IF it works on it, then you might actually expect a Block Viewer during the week because it will be really easy to program.

Second: If you want to read the System Area inside the disk with the Leo library from the SDK, then you might have to take a look at this:
http://pastebin.com/umq12QdJ

It does miracles. Thanks Zoinkity and OzOnE, what you gave me are really useful. That's my christmas present.

Third: I've done more documentation on the disk, thanks to OzOnE's Mario Artist Paint Studio dump. Unfortunately, it's incomplete, corrupted and pretty much a bad dump in itself. If he wants to release it, it's up to him. It's only good to look at with a Hex Viewer.
And there's lots of funny debug messages. I didn't even expect to laugh at them, but I might just share some of them with you guys:

- "FATAL ERROR: CAN'T CREATE N64DD MANAGER, EXPECT THINGS TO FUCK UP"
- "Trying to create an icon which allready has a back sprite index %s- ARE YOU FUCKING MAD ??"
- "oh shit, i can't sync up this icon!!"
- "Dog Shit Alley!"

EDIT:


So here's the first release of 64DD Block Viewer. It's made with the SDK, and it works... not on an Everdrive 64 v2.
It just doesn't work on it. It works on a HW v1, but that's it. I heard about using OS 1.27 and lower to get it working, but Jimmy130 and I couldn't find it, but anyway I've decided to give to you all, a development version:
https://mega.co.nz/#!dJRimJgR!1iqPx7TWcXuLmKpttbxe2_GZuy9SFFHVF-7-DkpFM2c

As a result, the future dumper will probably only work on a 64drive because of simpler librairies and inconsistencies with the Everdrive 64. I have no idea how to manually do that to a SD card anyway.
It doesn't work on a development 64DD, but here's a detail about blocks:
LBA 2,3,7,10,11,12,16,17,18,19,20,21,22 and 23 CANNOT BE READ. I assume it's a copy protection of the System Area.

If you see "Disk ID: ????", that means it doesn't recognize the disk.

If you see "LEO_ERROR_GOOD", don't worry, that means everything's fine, either way, the Hex Viewer will blink in red to warn you of an error. If it doesn't, you're good to go.
Also, to ensure it works, you must insert a disk BEFORE running the ROM. And then you can look at all the blocks of the disk. Feel free to manually type every bytes. lol

At least, be happy, dumping time should be soon enough. Maybe before 2015 if we're lucky.
I would want to release the source but it's based on a demo (pfs) which I only touched the main.c file.

 

I'm looking through the Wiki and looking at the proposed disk dump format and notice only LBAs 0 and 14 plan to be copied. While I imagine it's not (easily) possible to copy the LBAs that give a read error, I really think it would be a good idea to copy everything else, even if it's a duplicate, for the sake of preservation. Or at least add LBA 6 since it's not a duplicate.

EDIT: Also, are the ROM and RAM areas going to be one file or two? Unless there is some factory default data in the RAM that the ROM needs, I would suggest two different files to make confirming data in the ROM easier (or the option to dump them that way, but not by default).

 

The disk dump format is the one used by the SDK, I don't know if I will really use it, but I left it there for the sake of documentation.

My idea is to keep it only one file, and with emulation, we'll add another file with only the RAM data, copied from the dump and will be the one modified by the emulator. I like preserving original dumps. But anyway, depending on the disk type, the offset to the beginning of RAM data is pretty much fixed so it's not much of a problem.

 

Ah, okay, thanks for the clarifications. It's good to see there will be a separate RAM file created. Good work so far!

 

64DD Block Viewer 0.2:
https://mega.co.nz/#!lYYWACTZ!ep_exlTTGOWGsBdsyZtg2U4o2dj7LJ-N2gGcHE1fv_M

Changes:
- Added ASCII View
- Fixed LBA/Offset select (it won't go out of bounds anymore)
- And some few minor tweaks.

No sorry, it's not fixed for every flashcarts. I got reports that it only worked fine on an Everdrive 64 v1, which I don't really get. I'll try to understand why but that's not easy at all.

EDIT:
64DD Block Viewer 0.3a:
https://mega.co.nz/#!1EYmSZST!nz9pSoO-3lCGaaldai6eZ9W7dp_pgU2Bg_N0U31v580

Changes:
- Fixed everything.
- Now works everywhere.
- Added Drive Version and all that.

EDIT2:
I made a working dumper. Yep. It's working. It does not write to SD card yet but the dumping function is working. It's a matter of time.

 

So i have followed this thread a while because i wanted to see if there is even gonna be progress made, and man, i have to say, you are some abitious guys! I wouldnt have thought that i would see some kind of tool to actually dump anything from the 64DD.
count me in for debugging and testing, i own a 64DD with some games, an Everdrive (v2 i think), and some programming skills (yet i was only active for NES, SNES and GB, and i dont know if you even need anyone to help on programming ).
If you need me to test something just pm me, i would be glad to help

Your tool seems to work, i get some data from my mario artist paint studio to show, as a test i looked @lba 14 which holds the expected disk id.
keep up the work, i want to see a dumped disk run on an emulator

 

Get hype people, it's finally happening.

 

64DDdump FIRST RELEASE by yours truly and marshallh:
http://bsxproj.superfamicom.org/64dd/64dddump.z64

You need a 64drive to dump it!

- Put in a disk, run it, and press START to dump!
- READ ERRORS before LBA 24 are normal. Don't panic!!
- In case the write fails, you can use the USB port to dump the SDRAM.
- Only works on retail 64DDs for now.

 

This is amazing work both of you! Many thanks for all your hard work! I know CF is not recommended for a GB dumper that is available for the 64drive, is it the same here, or is either (CF/SD) okay to work with (no complaints here, just so I know)?

 

I don't think it works on CF. But marshallh knows more than me about this.
But it definitely works on SD, it's been extensively tested.

 

Works fine on the CF cards I have here. I think the problems Mike had with agbd were mostly due to the old firmware and a flaky CF

 

Cool.

Glad you got it going, guys.

Does it now work for all sectors past the 2140-ish LBA range without errors?

Would it be possible to still get the Everdrive 64 version working, 'cos I don't have a 64drive (sadly)?
EDIT: Although, I only have Mario Artist: Paint Studio, so it's maybe not a priority atm if it's working on the 64drive.

Also, is it the general consensus that the errors in the system area are intentional, and that it's just a fairly simple obfuscation?

This is really great though, as there are only so many disks to dump and it won't take too long.
Maybe this could help Jimmy to backup his SM64 disk as well?

OzOnE.

 

Errors at the System Area really seem intentional, it's the same errors over & over for several disks.
On RandnetDD dumping, everything went smoothly.
However on SimCity 64 dumping, at LBA around 2140, it did get errors. So I guess it's done on purpose, and not only with Mario Artist?

About the Everdrive 64... is it possible to dump the SDRAM with USB? I don't think we'll do SD card writing on it. It was a pain in the ass to get it working.