Question regarding the "Achilles heel" of the PS2

Okay so this may have been asked before, but here goes

I am well aware the Hardware of the PS2 is unchangeable. My question is this is it possible to use an internal HDD to extend the main RAM and VRAM of the console, I know that retail games would not benefit from this my question is more focused on home-brew, further question would this impart any performance gains and or allow more flexibility with a home-brew game?

 

Linux calls it a Swap, Windows calls it a Pagefile. Swapping access speed for "memory" has been done since forever - why not program to the hardware?

 

Secondary storage is much, much slower than memory. So if you involve page-swapping, you will lose performance.
I/O on the PlayStation 2 is also quite slow.

I used to wonder why they only put 4MB of VRAM, until @l_oliveira explained that the sort of memory they put in is actually part of the GS chip - very fast memory that is expensive.
Hence VRAM cannot be extended in any way because it is not directly manipulatable from the EE. Some people wrote that it is meant to be used as a texture cache, and I agree.

As for EE itself, there is the TLB, which is used by PS2Linux and some games for memory management. So memory management can be done there, but it might not be the best thing to do...

 

I see thank you for the info, this actually makes a future project of mine more manageable. I was just curious as to whether such a thing is possible now that I know it is but at the cost of performance I can adjust focus on other aspects, namely proper usage of the VU's (namely VU1 as I have read it is quite a nuisance to deal with) also thanks for the info about the VRAM I didn't realize the unique properties it possessed. 4MB may not be much but considering what I have seen done with the console I'd say the speed makes up for the lack of space. Speaking of I/O would a game playing entirely from the HDD have detrimental affects on performance for whatever reason ULaunchELF does not like the DVD Drive on my Phat that or I am using incorrect media/burn rates on said media. Plays retail discs just fine

 

Anyway, you're also constrained by the default resolutions of this console - NTSC and PAL. DTV 480P mode isn't actually the HD 480P mode, but one with a 3:2 resolution (720x480). So you may be using something like 640x448/640x512 on this console.
They only made these 3 modes available to licensed developers. The rest that we found, were obtained from reverse-engineering the kernels and from PS2Linux.

Some developers don't even use the full sizes of these resolutions - they use a smaller frame buffer, that gets upscaled with the hardware (MAGH & MAGV). So you can save even more memory that way.

You should make your game fit the characteristics of the device, which is why thread priorities also have to be managed and Sony usually gave developers the option to change the priorities of the Sony libraries.
The CD/DVD drive is operated with libcdvd, which also offers asynchronous functions that game developers usually use for accessing files. This cannot be done with the HDD unit.

When working on OPL, I have observed that the longer maximum transfer length of ATA could actually cause some games to suffer from performance problems, due to thread starvation.
For example, Dragon Quest 8 and Melty Blood would have noticeable stuttering, when there is a lot of HDD activity, due to the DEV9 driver doing polling on the DMA channel's status and CDVDFSV being more active due to ATA being able to fulfil requests much faster than the CD/DVD drive.
The solution here, IMO, would be to either make reading a lower priority than sound-streaming. Or to organize reading to take place at a better time. Or to actually use data streaming, rather than reading a huge chunk of data at once.

We use the Sony modules (CDVDMAN + CDVDFSV), so it probably isn't something that is software-specific.

You might just have a disc that cannot be read properly by your console, for one reason or another. You should use high-quality discs and follow good burning advice.
The goal is to get low jitter (noise), so that the drive can read properly. Don't just follow advice, particularly those that go along the lines of lower burning speed or using an old CD/DVD writer...

 

I see, rather interesting as I read somewhere in the GS Manual I believe the hardware is capable of 1080i ( at least I remember reading something like that but unable to confirm at the moment ) well if it helps when I boot up the console my TV reports 480P I believe in the corner, this is on a retail game and according to the above info this is something that only developers had access to.

So two routes stream the game/app through the ethernet port or use better quality disc than the bargain bin DVDR's I have been using. What it seems like for the project I have in mind it would be best to code the application to be read and execute in small sections rather than large chunks, that probably wasn't the best way to word that, but I am very much a novice at this point. That being said this information has been very informative particularly the info about the video modes

 

The manual does mention that it supports 480P and 1080I, but does not document how they are to be used.
To use a video mode, the SetGsCrt() syscall has to be invoked to configure the GS for that mode. Hence the EE kernel determines the video modes that can be selected, although it is still determined by software (i.e. we can bypass the kernel to add custom video modes). However, the Sony libgraph library only supported NTSC and PAL. Support for the 480P mode was only added at release 2.5, and other video modes were documented to be available on request (which is probably how arcade systems and some games do have access to more video modes, including 1080I).

So for any homebrew library to know the codes for the other video modes, its developer either referred to PS2Linux or did reverse-engineering.

I would say that they only added support for the practical video modes.

When I mentioned streaming, I don't mean you have to use Ethernet.
I meant that you might want to not read large blocks of data (e.g. everything) in at once - not only might be affect ongoing audio playback (for example), you don't have so much memory anyway. And it is not that making large reads certainly will not be good for performance, it's just that it will take up a chunk of CPU time, which can cause other time-sensitive tasks to be disrupted if you do not manage this properly.

The EE and IOP kernels use non-preemptive multitasking.

But well, I wouldn't be too concerned about this if I were you.

 

hmm very interesting, your post made me think if perhaps there is way to sort of "question" the EE kernel and have it report back the codes but I feel I am thinking very wishfully and with incomplete knowledge.
Also took some time to lightly educate myself on the term "non-preemptive multitasking" I can see why you say I shouldn't be too concerned but the more information I gleam and discover the easier it seems for me to figure out how I want this game(program) to work. I have the suspicion an open world environment will either be very difficult or impossible but the more I discover the more I want to try. Speaking of manuals though I probably know the answer will the main TX79 Core Architecture manual prove to be useful I haven't had the chance to truly crack it open but I feel it covers the system in a more broad aspect as broad as 700-ish pages can be I guess

 

That's why we reverse-engineer it, to see what functions were implemented and what values are set for each mode.
It's how the "1080P" mode was implemented with GSM - we copied the code for 1080I, increased the PLL input and disabled interlace.

It is good to design the system to work well with the scheduler's design from the very beginning. If you can learn how it works - then that is very good because it will allow you to properly utilize the scheduler's services.

I don't see why it would be impossible, although you will have to design a system that will fit with the PlayStation 2's hardware capabilities.
All the best to you with this project.

If you have not already obtained a copy of the Sony PS2SDK, you might want to consider obtaining a copy.

To properly utilize the PlayStation 2, you need to have access to its architecture-specific documentation. No other MIPS ever had two integer pipelines, VUs and MMI.
MMI is not widely used by our compiler because it has no support for vectors, so you might want to explore that as well. I saw that there are some MMI that were made to facilitate quick and efficient packing of GIF DMA packets.

Try to also be a master of the DMAC too, if you want performance. Be sure of the cache's design and the methods for dealing with cache coherency (uncached segment, ucab etc), and try not to make the mistakes I kept repeating when working on HDLGameInstaller.
While the DMAC for the EE requires addresses and sizes aligned to 16-byte boundaries, the cache lines are 64-byte in length. By performing DMA transfers at just any 16-byte-aligned address, normal access to adjacent buffers can corrupt the DMA transfer when the cache line is written back (either by eviction or by a deliberate writeback).

Finally, I know that getting an actual TOOL is expensive. So the next best thing is to use either RDB or Kermit.

 

after some reading I am under the impression the DMAC (Direct Memory Access Controller) is the roadway that interconncets all the components in the system while allowing direct access to the 32MB main RAM space For each device (I could also be either over-simplifying it or be incorrect in this assessment) Also is the 16-byte boundary specific to the EE?
RDB or Kermit, I am unfortunately unfamiliar with either term, does either refer to perhaps "jailbreaking" a retail console?
I have recently transferred a PS2SDK install to an XP machine and then recompiled it.
Though now considering the new info I have learned I fear the hardware of my "Dev computer" is insufficient for the task. And since I brought it up, in regard to 3d modeling what would you suggest as a tool to accomplish this task? I know having the PS2SDK is not enough so I was wondering of your personal input on software to accomplish 3d modeling, meshes and asset production

 

I personally use Blender for 3D modeling.

 

That sounds more like the data bus that the peripherals are connected to. From the notes of @wisi, it can be concluded that there are at least two busses in the PlayStation 2: the SBUS (EE and its peripherals) and SSBUS (IOP and its peripherals).
The DMAC performs data transfers on behalf of the processor.

There are a few DMACs in the PlayStation 2. The one that the EE has, connects only the EE peripherals (i.e. GS, scratchpad, IPU, IOP) to memory. Most of the I/O devices are connected to the IOP instead.
No device can initiate DMA transfers on their own, perhaps other than the i.Link and USB OHCI controllers.

Yes. If you ever study the DMACs on the IOP, you will find that it only requires 4-byte alignment.

No. They're just tools that add missing functionality to a CEX, to allow DECI2 to function for debugging. Both can be found on this forum.
There will be some limitations because the CEX has less memory than a TOOL, but that will still allow debugging.

Insufficient, in what way? Back in 1999, the developers were probably using Pentium II computers. You do not need much to compile and run your code, perhaps unless you deal with 3D modeling.

 

hehe, well the Dev machine is an old P4 single core with HT, poor thing was crying when I recompiled the PS2SDK, though for 12-ish dollars I can't complain with 2GB RAM and very basic (its an old Dell BTW) hmm, ya know at this point I might just keep scrounging around for a TOOL but seeing as that it is less an issue of price (couldn't afford it anyway) and more an issue of actually finding one. I will make due and since 3d modeling for this project is a must then yes I will have to beef up the machine, (IE convert my gaming rig to Development environment or build a new dev machine) I know that might sound silly but I like to keep the two separate as my Dev machine is purposefully not hooked up to the internet. Anyway I have much reading to do first round of homework is EE core followed by VU then GS, as the community has been gracious to supply much of the info I have procured I will be more than happy to repay that with my thoughts, analysis and any code I happen to generate for this project, that will take some time but I look forward to when that little stop in this road comes. Until then Assemblers happy coding and homebrewing.

 

You might want to get a good CPU and GPU so that you can debug in PCSX2. At least they are easier to find than the TOOL (despite the mining crisis)

 

Well with all the hardware I have lying around its just a matter of procuring a "new" cpu and motherboard plus ram been thinking now to revisit the phenom ii x6 and couple with an old nvidia card I have. Using PCSX2 to debug? hmm that's actually a good idea. Anyway Back to homework.

 

What benefits are there from debugging under PCSX2? Does it have a nice UI?
(No really, I never found out if it has a debug function, since I only used its console for handling printf)

Among the DSNET tools that are bundled with the official SDK, none offer a nice UI, which adds onto the complexity of the traditional setup.

 

after a bit of googling PCSX2 did have a debugger built in but later versions had it removed due to being dubbed "not useful"
No it did not have a UI, sucks I thought I would be able to use it to simplify my work. I only thought it was a good idea as a debugger because, A: it allows me to test builds on the computer and rather quickly. B: despite the fact that the last time I used it, it did not emulate 100% (at the time DX9 was still being used and the emulator was not 100% hardware accurate ) Well, sucks but still useful to have at least to test builds at least we still have Kermit and RDB maybe I can get those to work under PCSX2 and debug that way (doubt it)

 

It should still have a debugger.
Also, OpenGL renderer is getting more accurate, especially with Mesa speed.