The BIS storage interface is now initialized by utilsInitializeResources() via bisStorageInitialize(). It takes care of mounting all eMMC FAT partitions in read-only mode, making it possible for the rest of the code to interact with them at any time without having to manually mount them beforehand.
Other changes include:
* bis_storage: add bisStorageExit().
* bis_storage: add bisStorageGetGptPartitionNameByBisPartitionId(), bisStorageGetSystemInitializerPartitionNameByBisPartitionId() and bisStorageGetMountNameByBisPartitionId() helpers.
* bis_storage: change bisStorageMountPartition(), bisStorageUnmountPartition() and bisStorageUnmountAllPartitions() to static functions.
* bis_storage: mount FAT partitions with FatFs by using drive numbers instead of arbitrary strings.
* cert: update code to reflect BIS storage interface changes.
* defines: add BIS_FAT_PARTITION_COUNT macro.
* fatfs/ffconf: use BIS_FAT_PARTITION_COUNT macro as the value for the FF_VOLUMES macro.
* fatfs/ffconf: disable arbitrary string support for volume IDs.
* nxdt_devoptab: increase concurrent mounted device count to 8.
* nxdt_utils: add missing underscores to anonymous variables in SCOPED_LOCK and SCOPED_TRY_LOCK macros.
* nxdt_utils: take care of BIS storage interface (de)initialization.
* poc: update code to reflect BIS storage interface changes.
* tik: update code to reflect BIS storage interface changes.
Takes care of retrieving a FsStorage object for any FAT eMMC BIS partition, mounting it via FatFs and creating a virtual devoptab device that can be used to carry out FS operations. All write operations have been stubbed, disabled or ifdef'd out of the code.
Other changes include:
* cert: update code to use the new BIS storage interface.
* defines: remove BIS_SYSTEM_PARTITION_MOUNT_NAME macro.
* devoptab: slightly improve macros.
* devoptab: add operation table for FatFs devices.
* devoptab: add rodev_fstat().
* devoptab: add devoptabMountFatFsDevice().
* fatfs: update diskio code to use the new BIS storage interface.
* fatfs: update configuration.
* save: update code to use regular C I/O calls instead of FatFs calls.
* tik: update code to use the new BIS storage interface.
* utils: remove eMMC BIS System partition (un)mount code.
Other changes include:
* config: remove ConfigChecksumLookupMethod enum.
* config: remove "gamecard/checksum_lookup_method" integer field.
* config: add "gamecard/lookup_checksum" boolean field.
* config: update default configuration file.
* defines: remove all NSWDB macros.
* GameCardImageDumpTask: update class to reflect the changes made to the configuration interface.
* GameCardImageDumpOptionsFrame: update class to reflect the changes made to the configuration interface.
* OptionsTab: remove OptionsTabUpdateFileDialog class.
* OptionsTab: remove "Update NSWDB XML" element.
* i18n: update localization files to reflect the rest of the changes.
The directory layout is partially based on the C++ namespaces we're currently using.
Other changes include:
* devoptab: move directory into "core".
* fatfs: move directory into "core".
* GameCardTab: move portions of logic from PopulateList() into their own methods.
* GameCardTab: use a macro to generate the properties table.
* GameCardTab: use a macro to add ListItem elements.
* GameCardTab: update AddApplicationMetadataItems() method to also display the number of DLCs available in the inserted gamecard for each application whenever possible.
* Makefile: remove all extra entries from the INCLUDES variable.
* nxdt_includes: move HOS version structs into their own header file.
* tasks: move code for each individual task into its own file(s).
* title: update titleGetGameCardApplicationMetadataEntries() to also count the number of DLCs available in the inserted gamecard for any given base application.
* title: reorder gamecard application metadata entries by name before returning the buffer in titleGetGameCardApplicationMetadataEntries().
GameCardImageDumpTask is a derived class of DataTransferTask, and it's designed to dump a gamecard image using the options selected by the user (which must be passed from a GameCardImageDumpOptionsFrame object). It uses std::optional<std::string> as its return type -- the idea behind this is to return error strings that may later be displayed by an ErrorFrame during the dump process (views not yet implemented).
FileWriter is a class that encapsulates write operations to different storage mediums (SD card, USB host and UMS devices), based on the provided input path. It is used by GameCardImageDumpTask to painlessly write data to the right output storage without explicitly having to implement multiple code paths for all storage types as part of the actual dump code. Furthermore, FileWriter also supports writing split files to FAT-formatted UMS devices if an output file is >= 4 GiB -- part file handling is completely abstracted away from any callers.
Other changes include:
* AsyncTask: rename all class methods to use PascalCase naming.
* AsyncTask: rename get() -> GetResult().
* DataTransferTask: reflect the changes made to AsyncTask.
* DataTransferTask: pause the RepeatingTask right after LoopCallback() returns true instead of pausing it in the cancel/post-execute callbacks.
* DataTransferTask: add private FormatTimeString() method.
* DataTransferTask: remove superfluous override for DoInBackground() -- classes derived from DataTransferTask must provide it on their own, anyway.
* DataTransferTask: add public GetDurationString() method.
* defines: update FAT32_FILESIZE_LIMIT macro to use UINT32_MAX.
* defines: add CONCATENATION_FILE_PART_SIZE macro (used by the new FileWriter class).
* DownloadTask: reflect the changes made to AsyncTask.
* DumpOptionsFrame: file extension is no longer stored as a class member, nor required by the class constructor.
* DumpOptionsFrame: change the return type for GetOutputFilePath() to bool. The method now saves its output to a variable passed by reference.
* GameCardImageDumpOptionsFrame: reflect the changes made to DumpOptionsFrame.
* i18n: update localization strings where applicable.
* nxdt_utils: fix a potential buffer overflow in utilsGetFileSystemStatsByPath().
* OptionsTab: reflect the changes made to AsyncTask.
* usb: add const qualifier to the input buffer required by usbSendFileData().
* usb: add const qualifier to the input buffer required by usbSendNspHeader().
Other changes include:
* DumpOptionsFrame: update contructors to also take a base output path string.
* GameCardImageDumpOptionsFrame: simplify constructor by letting it take care of retrieving the title on its own.
* nxdt_utils: remove utilsCreateOutputDirectories() function -- we'll be using utilsCreateDirectoryTree() anyway, so it's okay.
Adds the DataTransferTask and DataTransferProgressDisplay classes, which use repurposed logic from the DownloadTask and EtaProgressDisplay classes, respectively. The EtaProgressDisplay class and EtaProgressInfo struct have been removed.
Other changes include:
* DownloadTask: heavily simplified logic by using DataTransferTask as the base class.
* DumpOptionsFrame: implement some additional/experimental changes.
* OptionsTab: update logic to use the new DataTransferProgressDisplay class.
* codebase: remove all references to Lockpick / Lockpick_RCM.
* keys: retrieve Atmosphère's key generation in keysLoadKeyset().
* keys: replace all "key_gen_val" references with "mkey_index".
* keys: move latest master key checks in keysReadKeysFromFile() to keysDeriveMasterKeys().
* keys: update latest master key checks to determine if nxdumptool's hardcoded master key vectors are outdated, using the key generation value from Atmosphère. If the master key vectors are outdated, and the newer master key(s) are not available, an error will be displayed.
* nxdt_utils: update logic in utilsReplaceIllegalCharacters() to replace consecutive illegal characters with a single underscore.
* nxdt_utils: move servicesGetExosphereApiVersion() to nxdt_utils as utilsGetExosphereApiVersion().
* nxdt_utils: define internal UtilsExosphereApiVersion struct, which is used to parse the output from utilsGetExosphereApiVersion().
* nxdt_utils: add utilsGetAtmosphereVersion(), utilsGetAtmosphereKeyGeneration() and utilsGetAtmosphereTargetFirmware() functions.
* services: remove servicesGetExosphereApiVersion().
* services: Atmosphère version is now retrieved via utilsGetAtmosphereVersion().
libnx now implements fsDeviceOperatorGetGameCardIdSet(), so I got rid of my own implementation.
Other changes include:
* cnmt: add cnmtVerifyContentHash().
* defines: add SHA256_HASH_STR_SIZE.
* fs_ext: add FsCardId1MakerCode, FsCardId1MemoryType and FsCardId2CardType enums.
* fs_ext: update FsCardId* structs.
* gamecard: change all package_id definitions from u64 -> u8[0x8].
* gamecard: fix misleading struct member names in GameCardHeader.
* gamecard: rename gamecardGetIdSet() -> gamecardGetCardIdSet().
* gamecard_tab: fix Package ID printing.
* gamecard_tab: add Card ID Set printing.
* host: add executable flag to Python scripts.
* keys: detect if we're dealing with a wiped eTicket RSA device key (e.g. via set:cal blanking). If so, the application will still launch, but all operations related to personalized titlekey crypto are disabled.
* pfs: rename PartitionFileSystemFileContext -> PartitionFileSystemImageContext and propagate the change throughout the codebase.
* pfs: rename PFS_FULL_HEADER_ALIGNMENT -> PFS_HEADER_PADDING_ALIGNMENT and update pfsWriteImageContextHeaderToMemoryBuffer() accordingly.
* poc: print certain button prompts with reversed colors, in the hopes of getting the user's attention.
* poc: NSP, Ticket and NCA submenus for updates and DLC updates now display the highest available title by default.
* poc: simplified output path generation for extracted NCA FS section dumps.
* poc: handle edge cases where a specific NCA from an update has no matching equivalent by type/ID offset in its base title (e.g. Fall Guys' HtmlDocument NCA).
* poc: implement NCA checksum validation while generating NSP dumps.
* romfs: update romfsInitializeContext() to allow its base_nca_fs_ctx argument to be NULL.
* usb: use USB_BOS_SIZE only once.
* workflow: update commit hash referenced by "rewrite-prerelease" tag on update.
Implement utilsParseHexString() and utilsAppletLoopDelay().
Other changes include:
* defines: add THIRTY_FPS_DELAY define.
* defines: remove unused CERT_PATH define.
* gamecard: restore original reserved_2 field in GameCardInfo struct. The last 16 bytes weren't being properly due to an issue within libnx's AES-CBC implementation.
* gamecard: rename GAMECARD_ACCESS_WAIT_TIME -> GAMECARD_ACCESS_DELAY.
* host: improve ABI documentation (thanks @v1993).
* keys: remove keysConvertHexDigitToBinary().
* keys: change keysParseHexKey() function signature + update it to use utilsParseHexString() instead.
* nso: change "NRO" to "NSO" in some misleading log error messages (thanks @liamadvance).
* nxdt_rw_poc: replace svcSleepThread() calls with utilsAppletLoopDelay().
* nxdt_utils: rename utilsGenerateHexStringFromData() -> utilsGenerateHexString().
* nxdt_utils: add static utilsConvertHexDigitToBinary() function, used by utilsParseHexString().
* nxdt_utils: update utilsAppendFormattedStringToBuffer() to avoid logging an error string if resources haven't been initialized.
* nxdt_utils: use a single call to sprintf() with a variable precision argument in utilsGenerateFormattedSizeString().
* nxdt_utils: call utilsAppletLoopDelay() in the while loop from utilsPrintConsoleError().
* nxdt_utils: remove unused utilsCloseFileDescriptor() function.
* tik: log input rights ID at the start of tikRetrieveTicketByRightsId().
And I don't need anyone to tell me so, thank you very much.
* PoC: remove gc_dumper and nsp_dumper PoC; create nxdt_rw_poc with all gc_dumper and nsp_dumper capabilities + standalone ticket dumping + raw NCA dumping; use ftruncate() to set output file sizes whenever possible. PoC code is a mess, as always. Expect the features from the rest of the PoCs to be implemented into nxdt_rw_poc soon.
* workflow: temporarily disable borealis build generation; comment out manual installation of up-to-date packages from Leseratte's mirrors because the latest devkitA64 Docker image has them all.
* borealis: update to fix building issues with latest devkitA64.
* bfttf: error out on invalid NCA signatures.
* config: save configuration to the current working directory; parse and validate new "gamecard/write_raw_hfs_partition" flag.
* defines: remove CONFIG_PATH macro; rename CONFIG_FILE_NAME.
* gamecard: rename fs_ctx -> hfs_ctx everywhere; use HFS function calls to retrieve partition names.
* hfs: move GameCardHashFileSystemPartitionType enum from gamecard.h and rename it to HashFileSystemPartitionType; add hfsIsValidContext(); add hfsGetPartitionNameString().
* nca/npdm: update comments to reflect latest HOS version.
* nxdt_bfsar: always generate absolute SD card paths with the device name; error out on an invalid NCA signature.
* nxdt_includes: include dirent.h; refactor Version struct to make it a union of all known *Version structs.
* nxdt_log: don't write session separator if the logfile is empty.
* nxdt_utils: log appletIsGamePlayRecordingSupported() errors; add utilsDeleteDirectoryRecursively().
* rsa: provide clearer function descriptions in header file.
* services: handle usb:ds initialization.
* tik: update tikConvertPersonalizedTicketToCommonTicket() to allow NULL input pointers as raw certificate chain arguments (much needed for standalone ticket dumping).
* title: add titleGetApplicationIdByMetaKey().
* usb: refactor interface (de)initialization code; slightly improve ABI usage (console-side only); redefine ABI version field in StartSession command blocks; upgrade ABI to v1.1.
* FatFs: rename DIR -> FDIR to avoid conflicts with definitions from stdlib's dirent.h.
* gamecard_tab: display package ID from the inserted gamecard; fix displayed version numbers from bundled system updates below 3.0.0.
* todo: add notes about creating devoptab devices for HFS/PFS/RomFS file tree dumping.
* Use forward declarations for type definitions wherever suitable (e.g. NcaContext, BucketTreeContext).
* Replace references to "DumpDestination" with "OutputStorage".
* Replace references to "append key area" with "prepend key area".
* Update LZ4 to v1.9.4.
* Update Material Icons font to 2022-08-01.
* RootView: change USB-related icons, reflect how many UMS devices are currently connected to the console, provide a way for child views to retrieve the populated UMS devices vector.
* Tasks: implement GetUmsDevices().
* Update borealis.
* Update libusbhsfs.
* Begin implementation of a DumpOptionsFrame class to display configurable options before starting a new dump process.
* bktr: rename bktrIsValidSubstorage() to bktrIsValidSubStorage(), use bktrIsValidSubStorage() in bktrInitializeCompressedStorageContext(), fixed a bug in bktrReadCompressedStorage() where Sparse substorages wouldn't be allowed.
* GamecardTab: push a DumpOptionsFrame object onto the view stack if the XCI option is clicked.
Other changes include:
* Codebase: update all references to FocusableLabel and FocusableTable constructors to match the changes in FocusableItem.
* FocusableTable: objects can now be highlighted, but the highlight background won't be drawn.
* http: set low speed limit and time values.
Other changes include:
* Codebase: move JSON parsing logic from config.c/h to nxdt_json.c/h.
* Codebase: replace all calls to localtime() with localtime_r() to guarantee thread-safety.
* Codebase: updated todo.txt.
* utils: implement utilsParseGitHubReleaseJsonData(), utilsFreeGitHubReleaseJsonData(), utilsGetApplicationUpdatedState() and utilsGetApplicationUpdatedState().
* utils: add extra logic to move the application's NRO to its proper path if the launch path isn't the right one (commented out at this moment).
* utils: add extra logic to replace the application's NRO at exit (commented out at this moment).
* defines: add DEVOPTAB_SDMC_DEVICE and tweak GitHub URL macros.
* DownloadTask: set percentage to 0 if the download size is unknown.
* DownloadTask: fix ETA string formatting.
* OptionsTab: repurpose OptionsTabUpdateFileDialogContent into OptionsTabUpdateProgress.
* OptionsTab: implement OptionsTabUpdateApplicationFrame.
* RootView: move date formatting into the static GetFormattedDateString() method.
* Makefile: use _GNU_SOURCE as part of CFLAGS to use strptime().
Other changes include:
* defines: added NON_MOVEABLE macro.
* AsyncTask: set class as non-moveable, too.
* RootView: provide an override for getDefaultFocus().
* StatusInfoTask: replace GetStatusInfoData with IsInternetConnectionAvailable().
Other changes include:
* AsyncTask: use a recursive mutex instead of atomics. Fixes build issues related to non-trivially-copyable elements.
* http: remove CURLOPT_ACCEPT_ENCODING option in httpPerformGetRequest().
Only supports GET requests, but that's more than enough for the time being.
Other changes include:
* OptionsTab: added an option to update the NSWDB XML (not implemented yet).
* config: add missing flag check in getters and setters.
* Move default socket initialization from utils.c to services.c.
* Thread-safe.
* Provides getter/setter functions for the data types used by nxdumptool's configuration.
* Each setter function writes the modified JSON configuration back to the SD card.
* Configuration is validated on interface initialization. If validation fails, a default JSON template is loaded from the application's RomFS and written back to the SD card.
Other changes:
* Implement directory creation.
* Moved more preprocessor definitions to defines.h.
* Replaced strtok() calls throughout the code with strtok_r() to guarantee thread-safety.
* core: implemented SCOPED_TRY_LOCK macro. Specific functions are now using it instead of SCOPED_LOCK to avoid potentially locking the Borealis UI.
* UI: implemented background tasks for Borealis, which call functions that now use SCOPED_TRY_LOCK.
