Since it seems that the Ren'Py PSVita Distribution Tool (GitHub link) lacks of documentation, I think it's worth sharing what I found and had to figure out, specially for non-PS Vita devs like me. It's not meant to be a tutorial, but if people find it useful as such, great! Also, I may edit the post later on as I test it more.
1 - The process seems to work better by using both Linux and Windows 10. I used Linux Mint host machine with Win10 virtual machine, but I'd imagine the opposite (Win10 host and Linux VM) would work too, or even Win10 plus Microsoft's Windows Subsystem for Linux (WSL). And I can't seem to get the Ren'Py tool to run on Wine any longer, and I can't get it to work on Win7 either.
2 - The Vita can be a prick when it comes to file properties, and Linux/WSL seems to be ideal for properly setting the LiveArea images, from what I understand of both Windows and Linux. I followed the steps at this older post from here on vitahacks, with the only issues I noticed being all images losing all colors and thus turning black and white, and the background image getting pixelated after installing the game on the Vita (still figuring out paramenters). But despite those issues, at least this way, people who try to install the game won't get an error "0x8010113D" at 99%.
3 - for running the porting tool on Win10, you need .NET 5.0.x Desktop Runtime. "5.0.x" because I used 5.0.0 last year and 5.0.17 this week, and both seemed to work. Also, Microsoft's download page for the runtime. No reboot needed, at least for 5.0.17, but maybe worth doing anyways, just to make sure everything is loaded properly.
4 - For the tool to be able to find the projects, from what I observed, the folder structure needs to be more specific than what is said in the GitHub page:
Ren'Py PSVita Distribution Tool/
└── Assets/
└── vita-mksfoex.exe
└── Ren'Py PSVita Distribution Tool.exe
└── pngquant.exe
└── game-you-want-to-port/
└── game/
└── sce_sys/
└── all other game file and folders
4.1 - The sce_sys folder is where the images from point 2 go, and you can compare it to the same folder inside the VPK for DDLC (GitHub link) to know if you did the file structure right (VPKs are just fancy ZIP files, and they may be extracted with tools such as unar and 7-Zip).
4.2 - I don't know if it's needed and I may try without it later, but I also copied the files from the Assets/ folder into the game's folder.
4.3 - Apparently, according to the DDLC page above, you can add data files later, but I still need to test how that works.
5 - Within the program, the list of valid projects it could detect will appear on the left. Click on the one you want. Then, fill the fields on the bottom right. Name doesn't seem to be too limited. ID and and update version are. ID must be 4 letters followed by 5 numbers, no two ways around (e.g. RNPY12345). Also be mindful to not use an ID already officially used or from another homebrew people would be likely to use. It seems update can be any combination of 2 numbers, one dot, and 2 more numbers (e.g. 01.19), but for the sake of identification, maybe it'd be good to have incremental numbers for each new update (e.g. first update is 01.00, and the second is 01.01). Then hit the bottom right button, the big one that mentions VPK (the program may hang for a second, so just wait).
5.1 - After done, the VPK will be in:
Ren'Py PSVita Distribution Tool/
└── dist/
└── project-name/
6 - (Optional) Before installing the VPK, install the NoSleep plugin so that you can turn the screen off without putting the Vita in sleep mode, and thus not affecting processes that need the Vita to be "awake" to work. Helps preserving the Vita screen longer, considering installing can take quite a while.
7 - About installing, it works just like you'd install any VPKs. Just beware the more small files there are in the VPK, the longer it takes for it to install, and most small files seem to be picked after the VPK is about 80~90% done, so it may take quite a while for the process to finish.
8 - Some games may be laggy (the Vita has 512 MB of RAM while some Ren'Py games can be very laggy even in their native systems with far more RAM), and also given the resolution in the Vita is not that big, if your game is lagging, you may want to get the original project, compress the assets, recompile the game for its native system and do the process of porting to the Vita again.
Hopefully, those observations can be helpful!