Removes all references to Compressed storages being used as substorages for Indirect storages.
I should have taken care of this in 0f1055c84e, but I guess I just forgot.
* codebase: remove all references to Lockpick / Lockpick_RCM.
* keys: retrieve Atmosphère's key generation in keysLoadKeyset().
* keys: replace all "key_gen_val" references with "mkey_index".
* keys: move latest master key checks in keysReadKeysFromFile() to keysDeriveMasterKeys().
* keys: update latest master key checks to determine if nxdumptool's hardcoded master key vectors are outdated, using the key generation value from Atmosphère. If the master key vectors are outdated, and the newer master key(s) are not available, an error will be displayed.
* nxdt_utils: update logic in utilsReplaceIllegalCharacters() to replace consecutive illegal characters with a single underscore.
* nxdt_utils: move servicesGetExosphereApiVersion() to nxdt_utils as utilsGetExosphereApiVersion().
* nxdt_utils: define internal UtilsExosphereApiVersion struct, which is used to parse the output from utilsGetExosphereApiVersion().
* nxdt_utils: add utilsGetAtmosphereVersion(), utilsGetAtmosphereKeyGeneration() and utilsGetAtmosphereTargetFirmware() functions.
* services: remove servicesGetExosphereApiVersion().
* services: Atmosphère version is now retrieved via utilsGetAtmosphereVersion().
Other changes include:
* poc: fix switching to a different title entry via L/R/ZL/ZR if the content counts don't match. Fixes issues with MGS Master Collection games.
Takes advantage of the recently implemented devoptab interfaces. Supports individual file dumping and multiple file/dir dumping via highlighting. May possibly need some additional testing. The code could potentially be improved, but I'll look into it at a later time -- I just want to release this now.
Other changes include:
* devoptab: comment out log messages.
* utils: add utilsGetDirectorySize() function.
* romfs: add hash bucket fields to RomFileSystemContext struct.
* romfs: remove current offset fields from RomFileSystemContext struct and all functions that relied on them.
* romfs: add romfsCalculateEntryHash().
* romfs: update romfsInitializeContext() to also load file/directory hash buckets.
* romfs: update romfsGetTotalDataSize() to make up for the removed functions.
* romfs: update romfsGetChild*EntryByName() functions to use hash bucket lookups.
Other changes include:
* devoptab: add missing exit macros to some FS operations across all interfaces.
* poc: update extractedRomFsReadThreadFunc() to make up for the removed RomFS functions.
This commit implements a devoptab wrapper for RomFS sections within NCAs, using code from romfs.c/h.
Other changes include:
* devoptab: add additional safety checks and some minor tweaks to both hfs_dev and pfs_dev calls.
Improve dirnext() calls in both PFS and HFS devoptab wrappers to make them also return bogus "." and ".." directory entries.
The "st_ino" field is now also being filled for all file entries.
Other changes include:
* libs: update libusbhsfs.
This commit implements a devoptab wrapper for Hash FS sections within gamecard images, using code from hfs.c/h.
Other changes include:
* poc: disable buffering on FILE objects via setvbuf().
* gamecard: fix crash in gamecardInitializeHashFileSystemContext() while calculating the size for the root Hash FS partition.
This commit implements a devoptab wrapper for Partition FS sections within NCAs, using code from pfs.c/h.
Other changes include:
* codebase: use NX_IGNORE_ARG macro where needed.
* hfs: slight tweaks to some of the static functions.
* pfs: slight tweaks to some of the static functions.
* pfs: use pfsIsValidContext() where needed.
* utils: update utilsInitializeResources() to use __system_argc and __system_argv variables from libnx.
* todo: update text file.
Other changes include:
* poc: add a "reset settings" option to the root menu.
* poc: add a message about using the controller sticks to scroll faster.
* poc: update missing ticket message to be more detailed.
* poc: add "retrieved" flag to MenuElementOption struct, which is reset while going back to a parent menu via menuResetAttributes().
* poc: fix index/count values in title selection screens.
`nsEnsureGameCardAccess()` has supposedly helped another user to fix the gamecard launch error after exiting their application, but it's not working under nxdumptool. I have yet to find out why.
Other changes include:
* host: rename windows_install_deps.py -> install_deps.py and update it to support multiple operating systems.
* host: restore assertions and fix most static analysis errors related to type aliases and checks.
* host: catch exceptions thrown by usb.core.find() in usbGetDeviceEndpoints().
* host: move code to unconditionally enable 32-bit paths under Windows to uiStartServer() and cliInitialize(), respectively.
* host: remove incomplete file if the ongoing transfer was cancelled and we're not dealing with a NSP file.
Implements both `StartExtractedFsDump` and `EndExtractedFsDump` commands into both nxdumptool and the Python host script.
Other changes include:
* poc: wake the write thread up if a preprocessing error occurs in all extracted FS dump functions. Fixes previously unhandled hangups.
* poc: verify current NCA hash before sending its last data chunk while dumping a NSP.
* host: update command handler to support CancelFileTransfer commands issued in-between SendFileProperties commands.
* host: update Markdown document.
* usb: use UsbCommandType_Count for the safety check in usbPrepareCommandHeader().
* usb: log both USB command header and command block whenever an error is reported by the host side.
libnx now implements fsDeviceOperatorGetGameCardIdSet(), so I got rid of my own implementation.
Other changes include:
* cnmt: add cnmtVerifyContentHash().
* defines: add SHA256_HASH_STR_SIZE.
* fs_ext: add FsCardId1MakerCode, FsCardId1MemoryType and FsCardId2CardType enums.
* fs_ext: update FsCardId* structs.
* gamecard: change all package_id definitions from u64 -> u8[0x8].
* gamecard: fix misleading struct member names in GameCardHeader.
* gamecard: rename gamecardGetIdSet() -> gamecardGetCardIdSet().
* gamecard_tab: fix Package ID printing.
* gamecard_tab: add Card ID Set printing.
* host: add executable flag to Python scripts.
* keys: detect if we're dealing with a wiped eTicket RSA device key (e.g. via set:cal blanking). If so, the application will still launch, but all operations related to personalized titlekey crypto are disabled.
* pfs: rename PartitionFileSystemFileContext -> PartitionFileSystemImageContext and propagate the change throughout the codebase.
* pfs: rename PFS_FULL_HEADER_ALIGNMENT -> PFS_HEADER_PADDING_ALIGNMENT and update pfsWriteImageContextHeaderToMemoryBuffer() accordingly.
* poc: print certain button prompts with reversed colors, in the hopes of getting the user's attention.
* poc: NSP, Ticket and NCA submenus for updates and DLC updates now display the highest available title by default.
* poc: simplified output path generation for extracted NCA FS section dumps.
* poc: handle edge cases where a specific NCA from an update has no matching equivalent by type/ID offset in its base title (e.g. Fall Guys' HtmlDocument NCA).
* poc: implement NCA checksum validation while generating NSP dumps.
* romfs: update romfsInitializeContext() to allow its base_nca_fs_ctx argument to be NULL.
* usb: use USB_BOS_SIZE only once.
* workflow: update commit hash referenced by "rewrite-prerelease" tag on update.
Implement utilsParseHexString() and utilsAppletLoopDelay().
Other changes include:
* defines: add THIRTY_FPS_DELAY define.
* defines: remove unused CERT_PATH define.
* gamecard: restore original reserved_2 field in GameCardInfo struct. The last 16 bytes weren't being properly due to an issue within libnx's AES-CBC implementation.
* gamecard: rename GAMECARD_ACCESS_WAIT_TIME -> GAMECARD_ACCESS_DELAY.
* host: improve ABI documentation (thanks @v1993).
* keys: remove keysConvertHexDigitToBinary().
* keys: change keysParseHexKey() function signature + update it to use utilsParseHexString() instead.
* nso: change "NRO" to "NSO" in some misleading log error messages (thanks @liamadvance).
* nxdt_rw_poc: replace svcSleepThread() calls with utilsAppletLoopDelay().
* nxdt_utils: rename utilsGenerateHexStringFromData() -> utilsGenerateHexString().
* nxdt_utils: add static utilsConvertHexDigitToBinary() function, used by utilsParseHexString().
* nxdt_utils: update utilsAppendFormattedStringToBuffer() to avoid logging an error string if resources haven't been initialized.
* nxdt_utils: use a single call to sprintf() with a variable precision argument in utilsGenerateFormattedSizeString().
* nxdt_utils: call utilsAppletLoopDelay() in the while loop from utilsPrintConsoleError().
* nxdt_utils: remove unused utilsCloseFileDescriptor() function.
* tik: log input rights ID at the start of tikRetrieveTicketByRightsId().
The host script can’t run on Python 3.9 because of [PEP 604][pep]-style annotations. Adding a future import can fix this. The `print_function` import is being removed since it is unnecessary on all releases of Python 3.
[pep]: https://peps.python.org/pep-0604/
* signature: add comments to SignatureType enum entries about the exact signing algorithms and padding schemes used.
* signature: rename signatureGetSigType() -> signatureGetTypeFromSignedBlob().
* signature: rename signatureIsValidSigType() -> signatureIsValidType().
* signature: rename signatureGetSigSize() -> signatureGetSigSizeByType().
* signature: rename signatureGetBlockSize() -> signatureGetBlockSizeByType().
* signature: rename signatureGetSig() -> signatureGetSigFromSignedBlob().
* signature: rename signatureGetPayload() -> signatureGetPayloadFromSignedBlob().
* signature: add signatureGetBlockSizeFromSignedBlob().
* cert: add more comments to the code.
* cert: update code to match signature interface changes.
* cert: add CERT_RSA_PUB_EXP_SIZE macro.
* cert: change public_exponent field in CertPublicKeyBlockRsa* structs from u32 to u8 array.
* cert: add size field to CertificateChain struct.
* cert: rename certGetCommonBlock() -> certGetCommonBlockFromSignedCertBlob.
* cert: rename certGetPublicKeySize() -> certGetPublicKeySizeByType().
* cert: rename certGetPublicKeyBlockSize() -> certGetPublicKeyBlockSizeByType().
* cert: rename certIsValidCertificate() -> certIsValidSignedCertBlob().
* cert: rename certGetSignedCertificateSize() -> certGetSignedCertBlobSize().
* cert: rename certGetSignedCertificateHashAreaSize() -> certGetSignedCertBlobHashAreaSize().
* cert: remove certGetPublicKey(), certGetPublicExponent() and certCalculateRawCertificateChainSize().
* cert: add certGetPublicKeyTypeFromCommonBlock(), certGetPublicKeyTypeFromSignedCertBlob(), certGetPublicKeySizeFromSignedCertBlob(), certGetPublicKeyBlockSizeFromSignedCertBlob(), certGetPublicKeyFromSignedCertBlob(), certGetPublicExponentFromSignedCertBlob(), certIsValidCertificate() (w/diff func sig), certGetCommonBlockFromCertificate(), certGetPublicKeyTypeFromCertificate(), certGetPublicKeySizeFromCertificate(), certGetPublicKeyBlockSizeFromCertificate(), certGetPublicKeyFromCertificate(), certGetPublicExponentFromCertificate() and certGetHashAreaSizeFromCertificate() functions.
* cert: avoid byteswapping the public key type value in multiple places -- it is now only being done in certGetPublicKeyTypeFromCommonBlock().
* cert: call certFreeCertificateChain() in _certRetrieveCertificateChainBySignatureIssuer() before attempting to retrieve the certificate chain.
* cert: other minor changes and corrections.
* tik: update code to match signature interface changes.
* tik: add missing comments to TikPropertyMask enum entries.
* tik: add key_generation, enc_titlekey_str and dec_titlekey_str fields to Ticket struct.
* tik: update tikRetrieveTicketByRightsId() to also take in a key_generation argument, instead of getting it from the rights ID (which could fail if it's using a key generation lower than HOS 3.0.1) or the key_generation field from the common ticket block (which could fail if the ticket has been tampered by certain tools).
* tik: rename tikGetCommonBlock() -> tikGetCommonBlockFromSignedTicketBlob().
* tik: change function signature for tikGetTicketSectionRecordsBlockSize().
* tik: rename tikIsValidTicket() -> tikIsValidSignedTicketBlob().
* tik: rename tikGetSignedTicketSize() -> tikGetSignedTicketBlobSize().
* tik: rename tikGetSignedTicketHashAreaSize() -> tikGetSignedTicketBlobHashAreaSize().
* tik: rename tikGetEncryptedTitleKeyFromTicket() -> tikGetEncryptedTitleKey().
* tik: add tikIsValidTicket() (w/diff func sig), tikGetCommonBlockFromTicket(), tikGetHashAreaSizeFromTicket(), tikFixTamperedCommonTicket(), tikVerifyRsa2048Sha256Signature() and tikDecryptVolatileTicket() functions. Ticket signature verification is only carried out for common tickets in tikFixTamperedCommonTicket().
* tik: change argument order in tikGetTicketEntryOffsetFromTicketList() and tikRetrieveTicketEntryFromTicketBin().
* tik: add TIK_COMMON_CERT_NAME and TIK_DEV_CERT_ISSUER macros.
* tik: use a scoped lock when calling tikRetrieveTicketFromEsSaveDataByRightsId().
* tik: simplify certificate chain retrieval steps in tikConvertPersonalizedTicketToCommonTicket() by always using the XS00000020 certificate.
* tik: wipe license_type and property_mask fields in tikConvertPersonalizedTicketToCommonTicket().
* tik: other minor changes and corrections.
Other changes include:
* keys: fix key generation checks in keysGetNcaKeyAreaKeyEncryptionKey() and keysGetTicketCommonKey().
* rsa: move core logic from rsa2048VerifySha256BasedPssSignature() into a new function: rsa2048VerifySha256BasedSignature().
* rsa: add rsa2048VerifySha256BasedPkcs1v15Signature() function.
