* Back to the origins: Make memory manager take guest PA rather than host address once again
* Direct mapping with alias support on Windows
* Fixes and remove more of the emulated shared memory
* Linux support
* Make shared and transfer memory not depend on SharedMemoryStorage
* More efficient view mapping on Windows (no more restricted to 4KB pages at a time)
* Handle potential access violations caused by partial unmap
* Implement host mapping using shared memory on Linux
* Add new GetPhysicalAddressChecked method, used to ensure the virtual address is mapped before address translation
Also align GetRef behaviour with software memory manager
* We don't need a mirrorable memory block for software memory manager mode
* Disable memory aliasing tests while we don't have shared memory support on Mac
* Shared memory & SIGBUS handler for macOS
* Fix typo + nits + re-enable memory tests
* Set MAP_JIT_DARWIN on x86 Mac too
* Add back the address space mirror
* Only set MAP_JIT_DARWIN if we are mapping as executable
* Disable aliasing tests again (still fails on Mac)
* Fix UnmapView4KB (by not casting size to int)
* Use ref counting on memory blocks to delay closing the shared memory handle until all blocks using it are disposed
* Address PR feedback
* Make RO hold a reference to the guest process memory manager to avoid early disposal
Co-authored-by: nastys <nastys@users.noreply.github.com>
* Add AddressTable<T>
* Use AddressTable<T> for dispatch
* Remove JumpTable & co.
* Add fallback for out of range addresses
* Add PPTC support
* Add documentation to `AddressTable<T>`
* Make AddressTable<T> configurable
* Fix table walk
* Fix IsMapped check
* Remove CountTableCapacity
* Add PPTC support for fast path
* Rename IsMapped to IsValid
* Remove stale comment
* Change format of address in exception message
* Add TranslatorStubs
* Split DispatchStub
Avoids recompilation of stubs during tests.
* Add hint for 64bit or 32bit
* Add documentation to `Symbol`
* Add documentation to `TranslatorStubs`
Make `TranslatorStubs` disposable as well.
* Add documentation to `SymbolType`
* Add `AddressTableEventSource` to monitor function table size
Add an EventSource which measures the amount of unmanaged bytes
allocated by AddressTable<T> instances.
dotnet-counters monitor -n Ryujinx --counters ARMeilleure
* Add `AllowLcqInFunctionTable` optimization toggle
This is to reduce the impact this change has on the test duration.
Before everytime a test was ran, the FunctionTable would be initialized
and populated so that the newly compiled test would get registered to
it.
* Implement unmanaged dispatcher
Uses the DispatchStub to dispatch into the next translation, which
allows execution to stay in unmanaged for longer and skips a
ConcurrentDictionary look up when the target translation has been
registered to the FunctionTable.
* Remove redundant null check
* Tune levels of FunctionTable
Uses 5 levels instead of 4 and change unit of AddressTableEventSource
from KB to MB.
* Use 64-bit function table
Improves codegen for direct branches:
mov qword [rax+0x408],0x10603560
- mov rcx,sub_10603560_OFFSET
- mov ecx,[rcx]
- mov ecx,ecx
- mov rdx,JIT_CACHE_BASE
- add rdx,rcx
+ mov rcx,sub_10603560
+ mov rdx,[rcx]
mov rcx,rax
Improves codegen for dispatch stub:
and rax,byte +0x1f
- mov eax,[rcx+rax*4]
- mov eax,eax
- mov rcx,JIT_CACHE_BASE
- lea rax,[rcx+rax]
+ mov rax,[rcx+rax*8]
mov rcx,rbx
* Remove `JitCacheSymbol` & `JitCache.Offset`
* Turn `Translator.Translate` into an instance method
We do not have to add more parameter to this method and related ones as
new structures are added & needed for translation.
* Add symbol only when PTC is enabled
Address LDj3SNuD's feedback
* Change `NativeContext.Running` to a 32-bit integer
* Fix PageTable symbol for host mapped
* Refactoring of KMemoryManager class
* Replace some trivial uses of DRAM address with VA
* Get rid of GetDramAddressFromVa
* Abstracting more operations on derived page table class
* Run auto-format on KPageTableBase
* Managed to make TryConvertVaToPa private, few uses remains now
* Implement guest physical pages ref counting, remove manual freeing
* Make DoMmuOperation private and call new abstract methods only from the base class
* Pass pages count rather than size on Map/UnmapMemory
* Change memory managers to take host pointers
* Fix a guest memory leak and simplify KPageTable
* Expose new methods for host range query and mapping
* Some refactoring of MapPagesFromClientProcess to allow proper page ref counting and mapping without KPageLists
* Remove more uses of AddVaRangeToPageList, now only one remains (shared memory page checking)
* Add a SharedMemoryStorage class, will be useful for host mapping
* Sayonara AddVaRangeToPageList, you served us well
* Start to implement host memory mapping (WIP)
* Support memory tracking through host exception handling
* Fix some access violations from HLE service guest memory access and CPU
* Fix memory tracking
* Fix mapping list bugs, including a race and a error adding mapping ranges
* Simple page table for memory tracking
* Simple "volatile" region handle mode
* Update UBOs directly (experimental, rough)
* Fix the overlap check
* Only set non-modified buffers as volatile
* Fix some memory tracking issues
* Fix possible race in MapBufferFromClientProcess (block list updates were not locked)
* Write uniform update to memory immediately, only defer the buffer set.
* Fix some memory tracking issues
* Pass correct pages count on shared memory unmap
* Armeilleure Signal Handler v1 + Unix changes
Unix currently behaves like windows, rather than remapping physical
* Actually check if the host platform is unix
* Fix decommit on linux.
* Implement windows 10 placeholder shared memory, fix a buffer issue.
* Make PTC version something that will never match with master
* Remove testing variable for block count
* Add reference count for memory manager, fix dispose
Can still deadlock with OpenAL
* Add address validation, use page table for mapped check, add docs
Might clean up the page table traversing routines.
* Implement batched mapping/tracking.
* Move documentation, fix tests.
* Cleanup uniform buffer update stuff.
* Remove unnecessary assignment.
* Add unsafe host mapped memory switch
On by default. Would be good to turn this off for untrusted code (homebrew, exefs mods) and give the user the option to turn it on manually, though that requires some UI work.
* Remove C# exception handlers
They have issues due to current .NET limitations, so the meilleure one fully replaces them for now.
* Fix MapPhysicalMemory on the software MemoryManager.
* Null check for GetHostAddress, docs
* Add configuration for setting memory manager mode (not in UI yet)
* Add config to UI
* Fix type mismatch on Unix signal handler code emit
* Fix 6GB DRAM mode.
The size can be greater than `uint.MaxValue` when the DRAM is >4GB.
* Address some feedback.
* More detailed error if backing memory cannot be mapped.
* SetLastError on all OS functions for consistency
* Force pages dirty with UBO update instead of setting them directly.
Seems to be much faster across a few games. Need retesting.
* Rebase, configuration rework, fix mem tracking regression
* Fix race in FreePages
* Set memory managers null after decrementing ref count
* Remove readonly keyword, as this is now modified.
* Use a local variable for the signal handler rather than a register.
* Fix bug with buffer resize, and index/uniform buffer binding.
Should fix flickering in games.
* Add InvalidAccessHandler to MemoryTracking
Doesn't do anything yet
* Call invalid access handler on unmapped read/write.
Same rules as the regular memory manager.
* Make unsafe mapped memory its own MemoryManagerType
* Move FlushUboDirty into UpdateState.
* Buffer dirty cache, rather than ubo cache
Much cleaner, may be reusable for Inline2Memory updates.
* This doesn't return anything anymore.
* Add sigaction remove methods, correct a few function signatures.
* Return empty list of physical regions for size 0.
* Also on AddressSpaceManager
Co-authored-by: gdkchan <gab.dark.100@gmail.com>
* Implement VCNT based on AArch64 CNT
Add tests
* Update PTC version
* Address LDj's comments
* Explicit size in encoding
* Tighter tests
* Replace SoftFallback with IR helper
Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>
* Reduce one BitwiseAnd from IR fallback
Based on popcount64b from https://en.wikipedia.org/wiki/Hamming_weight#Efficient_implementation
* Rename parameter and add assert
Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>
Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>
* Add Pmull_V Sse fast path only, both "8/16B -> 8H" and "1/2D -> 1Q" variants; with Test.
* Add Clmul fast path for the 128 bits variant.
* Small optimisation (save 60 instructions) for the Sse fast path about the 128 bits variant.
* Add slow path, both variants. Fix V128 Shl/Shr when shift = 0.
* A32: Add Vmull_I P64 variant (slow path); not tested.
* A32: Add Vmull_I_P8_P64 Test and fix P64 variant.
* Implement VFNMA.F<32/64>
* Update PTC Version
* Update Implementation & Renames & Correct Order
* Fix alignment
* Update implementation to not trigger assert
* Actually use the intrinsic that makes sense :)
* shader cache: Fix Linux boot issues
This rollback the init logic back to previous state, and replicate the
way PTC handle initialization.
* shader cache: set default state of ready for translation event to false
* Fix cpu unit tests
* Add Fmax/minv_V & S/Ushl_S Inst.s with Tests. Fix Maxps/d & Minps/d double zero sign handling. Allows better handling of NaNs.
* Optimized EmitSse2VectorIsNaNOpF() for multiple uses per opF.
* Add CRC32 A32 instructions.
* Fix CRC32 instructions.
* Add CRC intrinsic and fast path.
Loop is currently unrolled, will look into adding temp vars after tests are added.
* Begin work on Crc tests
* Fix SSE4.2 path for CRC32C, finialize tests.
* Remove unused IR path.
* Fix spacing between prefix checks.
* This should be Src.
* PTC Version
* OpCodeTable Order
* Integer check improvement. Value and Crc can be either 32 or 64 size.
* This wasn't necessary...
* If size is 3, value type must be I64.
* Fix same src+dest handling for non crc intrinsics.
* Pre-fix (ha) issue with vex encodings
* Add Vmvn (register), tests for both Vmvn variants.
* Add Vpmin, Vpmax, improve Non-FastFp accuracy for Vpadd
* Rebase on top of PTC.
* Add Nopcode
* Increment PTC version.
* Fix nits.
* Delete DelegateTypes.cs
* Delete DelegateCache.cs
* Add files via upload
* Update Horizon.cs
* Update Program.cs
* Update MainWindow.cs
* Update Aot.cs
* Update RelocEntry.cs
* Update Translator.cs
* Update MemoryManager.cs
* Update InstEmitMemoryHelper.cs
* Update Delegates.cs
* Nit.
* Nit.
* Nit.
* 10 fewer MSIL bytes for us
* Add comment. Nits.
* Update Translator.cs
* Update Aot.cs
* Nits.
* Opt..
* Opt..
* Opt..
* Opt..
* Allow to change compression level.
* Update MemoryManager.cs
* Update Translator.cs
* Manage corner cases during the save phase. Nits.
* Update Aot.cs
* Translator response tweak for Aot disabled. Nit.
* Nit.
* Nits.
* Create DelegateHelpers.cs
* Update Delegates.cs
* Nit.
* Nit.
* Nits.
* Fix due to #784.
* Fixes due to #757 & #841.
* Fix due to #846.
* Fix due to #847.
* Use MethodInfo for managed method calls.
Use IR methods instead of managed methods about Max/Min (S/U).
Follow-ups & Nits.
* Add missing exception messages.
Reintroduce slow path for Fmov_Vi.
Implement slow path for Fmov_Si.
* Switch to the new folder structure.
Nits.
* Impl. index-based relocation information. Impl. cache file version field.
* Nit.
* Address gdkchan comments.
Mainly:
- fixed cache file corruption issue on exit; - exposed a way to disable AOT on the GUI.
* Address AcK77 comment.
* Address Thealexbarney, jduncanator & emmauss comments.
Header magic, CpuId (FI) & Aot -> Ptc.
* Adaptation to the new application reloading system.
Improvements to the call system of managed methods.
Follow-ups.
Nits.
* Get the same boot times as on master when PTC is disabled.
* Profiled Aot.
* A32 support (#897).
* #975 support (1 of 2).
* #975 support (2 of 2).
* Rebase fix & nits.
* Some fixes and nits (still one bug left).
* One fix & nits.
* Tests fix (by gdk) & nits.
* Support translations not only in high quality and rejit.
Nits.
* Added possibility to skip translations and continue execution, using `ESC` key.
* Update SettingsWindow.cs
* Update GLRenderer.cs
* Update Ptc.cs
* Disabled Profiled PTC by default as requested in the past by gdk.
* Fix rejit bug. Increased number of parallel translations. Add stack unwinding stuffs support (1 of 2).
Nits.
* Add stack unwinding stuffs support (2 of 2). Tuned number of parallel translations.
* Restored the ability to assemble jumps with 8-bit offset when Profiled PTC is disabled or during profiling.
Modifications due to rebase.
Nits.
* Limited profiling of the functions to be translated to the addresses belonging to the range of static objects only.
* Nits.
* Nits.
* Update Delegates.cs
* Nit.
* Update InstEmitSimdArithmetic.cs
* Address riperiperi comments.
* Fixed the issue of unjustifiably longer boot times at the second boot than at the first boot, measured at the same time or reference point and with the same number of translated functions.
* Implemented a simple redundant load/save mechanism.
Halved the value of Decoder.MaxInstsPerFunction more appropriate for the current performance of the Translator.
Replaced by Logger.PrintError to Logger.PrintDebug in TexturePool.cs about the supposed invalid texture format to avoid the spawn of the log.
Nits.
* Nit.
Improved Logger.PrintError in TexturePool.cs to avoid log spawn.
Added missing code for FZ handling (in output) for fp max/min instructions (slow paths).
* Add configuration migration for PTC
Co-authored-by: Thog <me@thog.eu>
* Implement a new physical memory manager and replace DeviceMemory
* Proper generic constraints
* Fix debug build
* Add memory tests
* New CPU memory manager and general code cleanup
* Remove host memory management from CPU project, use Ryujinx.Memory instead
* Fix tests
* Document exceptions on MemoryBlock
* Fix leak on unix memory allocation
* Proper disposal of some objects on tests
* Fix JitCache not being set as initialized
* GetRef without checks for 8-bits and 16-bits CAS
* Add MemoryBlock destructor
* Throw in separate method to improve codegen
* Address PR feedback
* QueryModified improvements
* Fix memory write tracking not marking all pages as modified in some cases
* Simplify MarkRegionAsModified
* Remove XML doc for ghost param
* Add back optimization to avoid useless buffer updates
* Add Ryujinx.Cpu project, move MemoryManager there and remove MemoryBlockWrapper
* Some nits
* Do not perform address translation when size is 0
* Address PR feedback and format NativeInterface class
* Remove ghost parameter description
* Update Ryujinx.Cpu to .NET Core 3.1
* Address PR feedback
* Fix build
* Return a well defined value for GetPhysicalAddress with invalid VA, and do not return unmapped ranges as modified
* Typo
* Implement VMOVL and VORR.I32 AArch32 SIMD instructions
* Rename <dt> to <size> on test description
* Rename Widen to Long and improve VMOVL implementation a bit
* Implement SMULWB, SMULWT, SMLAWB, SMLAWT, and add tests for some multiply instructions
* Improve test descriptions
* Rename SMULH to SMUL__
* Add SSAT, SSAT16, USAT and USAT16 ARM32 instructions
* Fix new tests
* Replace AND 0xFFFF with 16-bits zero extension (more efficient)
* Implement TEQ and MOV (Imm16)
* Initial work on A32 instructions + SVC. No tests yet, hangs in rtld.
* Implement CLZ, fix BFI and BFC
Now stops on SIMD initialization.
* Exclusive access instructions, fix to mul, system instructions.
Now gets to a break after SignalProcessWideKey64.
* Better impl of UBFX, add UDIV and SDIV
Now boots way further - now stuck on VMOV instruction.
* Many more instructions, start on SIMD and testing framework.
* Fix build issues
* svc: Rework 32 bit codepath
Fixing once and for all argument ordering issues.
* Fix 32 bits stacktrace
* hle debug: Add 32 bits dynamic section parsing
* Fix highCq mode, add many tests, fix some instruction bugs
Still suffers from critical malloc failure 😩
* Fix incorrect opcode decoders and a few more instructions.
* Add a few instructions and fix others. re-disable highCq for now.
Disabled the svc memory clear since i'm not sure about it.
* Fix build
* Fix typo in ordered/exclusive stores.
* Implement some more instructions, fix others.
Uxtab16/Sxtab16 are untested.
* Begin impl of pairwise, some other instructions.
* Add a few more instructions, a quick hack to fix svcs for now.
* Add tests and fix issues with VTRN, VZIP, VUZP
* Add a few more instructions, fix Vmul_1 encoding.
* Fix way too many instruction bugs, add tests for some of the more important ones.
* Fix HighCq, enable FastFP paths for some floating point instructions
(not entirely sure why these were disabled, so important to note this
commit exists)
Branching has been removed in A32 shifts until I figure out if it's
worth it
* Cleanup Part 1
There should be no functional change between these next few commits.
Should is the key word. (except for removing break handler)
* Implement 32 bits syscalls
Co-authored-by: riperiperi <rhy3756547@hotmail.com>
Implement all 32 bits counterparts of the 64 bits syscalls we currently
have.
* Refactor part 2: Move index/subindex logic to Operand
May have inadvertently fixed one (1) bug
* Add FlushProcessDataCache32
* Address jd's comments
* Remove 16 bit encodings from OpCodeTable
Still need to catch some edge cases (operands that use the "F" flag) and
make Q encodings with non-even indexes undefined.
* Correct Fpscr handling for FP vector slow paths
WIP
* Add StandardFPSCRValue behaviour for all Arithmetic instructions
* Add StandardFPSCRValue behaviour to compare instructions.
* Force passing of fpcr to FPProcessException and FPUnpack.
Reduces potential for code error significantly
* OpCode cleanup
* Remove urgency from DMB comment in MRRC
DMB is currently a no-op via the instruction, so it should likely still
be a no-op here.
* Test Cleanup
* Fix FPDefaultNaN on Ryzen CPUs
* Improve some tests, fix some shift instructions, add slow path for Vadd
* Fix Typo
* More test cleanup
* Flip order of Fx and index, to indicate that the operand's is the "base"
* Remove Simd32 register type, use Int32 and Int64 for scalars like A64 does.
* Reintroduce alignment to DecoderHelper (removed by accident)
* One more realign as reading diffs is hard
* Use I32 registers in A32 (part 2)
Swap default integer register type based on current execution mode.
* FPSCR flags as Registers (part 1)
Still need to change NativeContext and ExecutionContext to allow
getting/setting with the flag values.
* Use I32 registers in A32 (part 1)
* FPSCR flags as registers (part 2)
Only CMP flags are on the registers right now. It could be useful to use
more of the space in non-fast-float when implementing A32 flags
accurately in the fast path.
* Address Feedback
* Correct FP->Int behaviour (should saturate)
* Make branches made by writing to PC eligible for Rejit
Greatly improves performance in most games.
* Remove unused branching for Vtbl
* RejitRequest as a class rather than a tuple
Makes a lot more sense than storing tuples on a dictionary.
* Add VMOVN, VSHR (imm), VSHRN (imm) and related tests
* Re-order InstEmitSystem32
Alphabetical sorting.
* Address Feedback
Feedback from Ac_K, remove and sort usings.
* Address Feedback 2
* Address Feedback from LDj3SNuD
Opcode table reordered to have alphabetical sorting within groups,
Vmaxnm and Vminnm have split names to be less ambiguous, SoftFloat nits,
Test nits and Test simplification with ValueSource.
* Add Debug Asserts to A32 helpers
Mainly to prevent the shift ones from being used on I64 operands, as
they expect I32 input for most operations (eg. carry flag setting), and
expect I32 input for shift and boolean amounts. Most other helper
functions don't take Operands, throw on out of range values, and take
specific types of OpCode, so didn't need any asserts.
* Use ConstF rather than creating an operand.
(useful for pooling in future)
* Move exclusive load to helper, reference call flag rather than literal 1.
* Address LDj feedback (minus table flatten)
one final look before it's all gone. the world is so beautiful.
* Flatten OpCodeTable
oh no
* Address more table ordering
* Call Flag as int on A32
Co-authored-by: Natalie C. <cyuubiapps@gmail.com>
Co-authored-by: Thog <thog@protonmail.com>
