In 2016, the LBP PSP servers were shut down for good with the loss of all online features and the deletion of all uploaded user-created content. Sadly for us, no packet captures from before the shutdown emerged since no one seems to have cared about LBP PSP enough to try to preserve the servers before they were gone.
In 2021, the servers for all the PS3 games were also shut down, eventually leading to the development of Project Lighthouse. This was a re-implementation of the PS3/Vita servers in C#, and as I was friends with the at-the-time lead developer, I decided to throw my hat into the ring and try to implement PSP support into Lighthouse.
As there were no packet captures, I would be on my own trying to reverse engineer the networking protocol, but before that, I needed to get the game to ping my custom server and attempt to connect. I extracted the EBOOT.BIN from a dumped ISO, and started looking through it for URL strings I could modify, and...
Bingo. This is exactly what I wanted, straight URLs, almost identical to the PS3 games, easy to modify to point to a local address. I patched the EBOOT, used UMDGen to re-pack it into a valid PSP UMD ISO file, loaded it up on my PSP, and... nothing. The game got past the PSN login sequence, and a message appeared, stating that it failed to connect to the server, no request was ever received by me.
Now, this wasn't entirely unexpected, I was told already that it refused to connect to any server, even after a simple patch. Now I just had to figure out why. I had my inklings it was on the DNAS service, which is the way the PSP verified that your copy of the game was legitimate, it would send some information to the server, and the server would tell you whether or not it was okay.
I set up mitmproxy, a simple HTTP proxy that logs and traces all traffic going through it. I opened the game, and when I tried to go online this time, I got a different error, this one saying "DNAS authentication failed". Now this was interesting, as all that changed was I set up an HTTP proxy between my PSP and the internet. After poking at it for another day or two, I had assumed that there was some "anti-HTTP proxy" code in place for DNAS, to prevent the kind of snooping I was trying to do. I gave up, and let the idea rest for a while. Some others working on Lighthouse had tried before and after me, but none made any more progress.
Nearing a year ago, the lead developer of Project Lighthouse split off from LBPUnion to create LittleBigRefresh, a new LittleBigPlanet custom server implementation, and I joined along for the ride early on. And on the first day of September 2023, I had a lot of free time and was willing to give LBP PSP another shot.
This time I wouldn't be giving up so easily. I dumped the latest release of the game and started picking it apart. I searched through the EBOOT.BIN file for the URLs I had seen before, and could not find them. After a little more searching, I found out the networking code had changed from the initial release to the latest update (2.0.5), and the URLs were stored differently.
Rather than the URLs being stored in their raw form, they were stored in composite parts, with the domain being formatted into a buffer. No problem, it was clear how it went together, and I soon had patched the URLs and was ready to test. This time I took a different approach, before testing on real hardware, I looked to see if PPSSPP had Infrastructure networking support, and to my surprise, there was a PR opened 3 days before adding exactly what I needed.
I rebased this PR on top of PPSSPP master and started testing. The first roadblock was the game failing to ping a server at all, and unimplemented function warnings spamming the console log. After realizing I was using the wrong ISO, I hit the first of many bugs in PPSSPP that were necessary for the game to connect.
HLE/sceParseUri.cpp:48 ffffffff=sceUriParse(08ed889c, http://[ip_redacted]:10061/LITTLEBIGPLANETPSP_XML/login?applicationID=22234&languageID=1, 08ed69b0, 00000000, 101): invalid arg
After patching PPSSPP to allow a null pointer for the 4th argument, it made it past that, and reached
sceNetResolverStartNtoA, which had not been implemented yet. Stubbing out more and more functions,
it eventually reached this nasty error:
Net/URL.cpp:13 Invalid URL: /LITTLEBIGPLANETPSP_XML/login?applicationID=22234&languageID=1
I eventually traced this back to a missing sceNetResolver implementation, and various missing
sceNetInet functions. A temporary workaround was to always prepend
http://localhost:10061 to every URL it parsed. Silly, yes, but it got me further. Next, it was
opening a connection, but not sending any data, leading to my simple listen server throwing up a weird error.
After a lot of frustration, I tried making all PPSSPP networking synchronous, and... that fixed it! Not sure why that helped, but it's making HTTP requests now, and I'm not one to complain.
Next, all I had to do was start up Refresh and get this baby rolling! Spun up the server, added the PSP title IDs so the server could recognize it, started up PPSSPP, went online, and-
Huh, that's not the kind of error I was expecting. After some intense debugging turns out it was a classic
Microsoft API, making our lives harder for no reason. LBP PSP sends duplicate host HTTP headers,
HOST, and Host, and .NET was combining the values of those headers into
localhost,localhost because... reasons.
One small patch to our HTTP server later and finally things were happening! LBP PSP had just sent its first-ever login request, likely since the shutdown. All I needed to do was create an account with the name PPSSPP, and the server should let the player in!
I created an account, went to log in, the server received the request, accepted it, and returned a
200 OK request to the client... and the game crashed. Not entirely unexpected, it crashed in the
"svo login" thread, so I assumed the wrong data was being sent with the login response.
Checking all the data being sent back and forth, I saw an interesting set of HTTP headers the game was looking
for: X-exe-v and X-data-v. I figured the game probably wanted me to just return these
to it as is, so I wrote a small middleware into the server to forward them back to clients, and...
It worked! The game showed the online EULA screen. And ended up making a total of 3 HTTP requests, one to log in, one to get recent news, and one to get the announcement text.
Interestingly the game failed to authenticate for the announce endpoint, after looking at the data, it was failing to send the authentication ticket to the server. This was quickly realized to be a likely response formatting error, and looking at the strings in the EBOOT for hints, jvyden found the PSP game expects a different login response body than the PS3/Vita games.
<loginResult>
<authTicket>TICKET_HERE</authTicket>
<lbpEnvVer>Refresh</lbpEnvVer>
</loginResult><authTicket>TICKET_HERE</authTicket>
After making the small
code change necessary, and reconnecting to the server, it finally logged in, tried to get
the news, got the announcement text, and showed the EULA. Scrolling through it, and clicking OK, I
was finally presented with the announcement text, meaning the game had fully authenticated and logged in. In
only 2 days we had gotten the login sequence working! I was in completely uncharted territory.
Going further caused the game to lock up, but after rebasing off some new commits in the PPSSPP sceHttp implementation, I successfully logged in and was able to see the community moon in all its glory. From here I was able to test some things out, notably, I was able to fully upload a level to the server, with few changes server-side.
The next step was browsing levels, or slots as it's known by the game. Clicking on "Community
Levels" leads to the game trying to get the leaderboards for the story levels... then immediately crashing
because we were returning a 404 Not Found. One
stub coming right up, send a blank string and a 200 OK, some more endpoint stubs and
shenanigans later, and...
There are the levels I uploaded! It all "just worked" and I was even able to download one on the first try! I tested the various filtering settings and most things seemed to work as intended with little server changes. Woo for code re-use!
Now was finally the time to put my head down and get real hardware working, I had developed a game plan and just needed to start working. I put the PSP through mitmproxy once again and got the same DNAS error as last time. This time I RTFM and realized that mitmproxy was blocking the DNAS request due to it using SSLv3, and the Arch Linux OpenSSL package did not have SSLv3 support enabled. I had tried to recompile OpenSSL, but struggled heavily, and eventually decided to give up on mitmproxy and ask around a bit for assistance.
In a previous
attempt, the Lighthouse team (including Dagg, Slendy, Penguins, Kairos, and Zhaxxy) had
figured out that the PSP was making a request to a PSN endpoint called imgw, which was used for
getting the user's friends list. I verified this was the case using tcpdump on my router. Since
this endpoint has long since been shut down, I needed to find a way around it.
I tried using a custom DNS server to re-route all imgw traffic to my server, but due to the same SSLv3 issue, while I was getting the traffic redirected to me, that went nowhere. After decompiling the game and RE a bit, I figured out that the calls responsible for calling imgw were syscalls prefixed with the name "sceNpRoster". I tried to write a PSP kernel module to patch those syscalls but had no success. Eventually, I just wrote a small 2-byte game patch to skip the branch that called the sceNpRoster family of functions and low-and-behold... nothing.
After setting up tcpdump on my router I saw that it was making a DNS query to
192.168.69.100, the IP of my desktop. At first, I thought this was due to the DNS setting, so I set
my PSP back to the 1.1.1.1 DNS server, and it still wouldnt connect to my server.
Then it hit me, like a brick covered in lemon juice. I collapsed on the floor live on a voice call once I
realized what was happening. The game was making a DNS query for 192.168.69.100 because that is
what the game was patched to. It didn't care that it was an IP, it was hardcoded to manually resolve the
domain it was patched to. I immediately re-patched the game to a domain pointing to my IP, loaded up OBS,
started recording, and recorded the live moment where
I connected real hardware to an LBP PSP server, for the first time in 7 years.
It's hard to see in the screenshot, but the game had connected and was showing the announcement text of the server.
Success! At least at first. The game crashed when attempting to browse levels. I left that aside though and started working on the easier tasks. I implemented grief reporting (eg. in-game reporting of offensive content), user leaderboards, and even got level uploads from real hardware working!
The LBP games use something called a "Digest key" to verify the integrity of data sent back and forth over the network, this is most likely to prevent tampering with the data and forged requests. This key is a very simple hash of the data body being sent, and the PS3/Vita algorithm has been known for many years. However, the PSP seemed to have a different digest algorithm and seemingly was getting mad at the digest I was sending it. The server was failing to verify the client digests too, although Refresh does not consider that a failure condition.
After trying and failing to RE the digest code myself, I was directed to one of the LBP wizards, ennuo, to ask for help. In under a day, they had figured out exactly what was going wrong and also instructed me to do one extra patch to get the game to be fully happy that we were going over HTTP only.
The PS3/Vita digest code would write the authentication ticket, the URL, and the digest key to a buffer, then
compute a SHA1 of that buffer. The PSP game would also add the numerical values of X-exe-v and
X-data-v to the buffer. Fixing
that discrepancy (along with the HTTPS patch), finally made the client fully happy with leaderboards.
Next was the final big roadblock, browsing levels on real hardware. This was quite a bug, since it only occurred on real hardware, I was unable to debug it using PPSSPP's built-in debugger. After a couple of days, I was able to recreate the crash on PPSSPP, and after failing to figure out the problem in Ghidra, I sent the stack trace over to ennuo for help, and very quickly they figured it out.
The PS3 games were already filled with networking spaghetti, but the PSP game really takes the cake, and
probably your girlfriend too, it's just kind of rude in general. LBP PSP was not accepting my slots
response because my XML fields were in the wrong order.
huh?
"Isn't XML usually parsed out into a tree structure in memory, and that tree structure is then indexed using
keys?"
It usually is, but LBP PSP instead iterates over all tokens in order they appear in the XML, and if it
knows what field that is, handles it. The handling code for one of the fields depended on the ID of the slot to
be read already, and since it was not read yet, the game was attempting to use that ID, dereferencing a bad
pointer and crashing. I had strict memory checking disabled in PPSSPP, which is why it was not having this crash
at first.
After making the server return the ID field first, it finally all came together and real hardware was finally able to browse levels! From there downloading levels worked fine, grief reports worked, and only a couple of patches later, user leaderboards and Vita worked through Adrenaline too!
After some more cleanup, PSP support was upstreamed into Refresh, and now I needed to make a process for patching retail copies since I can't go around sharing full EBOOTs or ISO files. For PS3 games I had helped develop a tool called Refresher. It can decrypt, patch, and then re-encrypt retail copies of LBP1/2/3/Vita, Sound Shapes, and any PS3/Vita game that uses raw URLs in its EBOOT for networking. I considered using the same tech for patching LBP PSP, although there were a few problems with this approach.
That second point immediately made things difficult enough to rule out normal patching. The only tool I know of that can make valid PSP ISO files is UMDGen, a Windows-only tool with no command-line interface. Not great for a cross-platform automated patching utility.
While I had been working on the server though, I had thought up a plan to remove the need to externally patch
the game at all. I wrote a small PSP kernel module to patch certain memory addresses when the game opened,
reading URLs from a .txt file. This means you wouldn't need to re-run the patching utility to
switch servers, all you needed to do was change out the text in a file. This was eventually moved into the
LittleBigRefresh GitHub org as Allefresher.
Allowing patching at launch time means you can now connect to custom servers using an original UMD
copy of the game, with no dumping required, which lowers the barrier to entry immensely.
After all of that mess, what did I learn? Once again I was taught to just ask for help jesus christ. This wouldn't have taken 3 full weeks of my time if I had asked for help earlier when stuck. The actual code changes needed server-side for PSP support were minimal, most of the effort was spent learning what needed to be done. Without help from people like jvyden, ennuo, and the previous work done by the Lighthouse team, this could not have been possible. Massive thanks to all of them!
Once the next Refresh update is released, I will start looking for testers, and if the server seems stable for a handful of days, we will make the announcement and write some docs for how to join. After release, I will clean up Allefresher and start looking into replacing the now-broken friends features. Once PPSSPP Infrastructure support is stable, I will also add PPSSPP support to Allefresher. This would allow emulator players to join, but at the moment it is too unstable to release and likely will be for a long while. It's possible Jpcsp will work using a patched ISO, but I was unable to get Jpcsp working on my machine, so that avenue is untested.