Commit graph

142 commits

Author SHA1 Message Date
FICTURE7
0d23504e30
Fix PTC count table relocation patching (#2666)
Fix an issue introduced in #2190 where by 2 different count table entry
addresses were used for LCQ functions. E.g:

```asm
 .L1:
   mov rbp,COUNT_TABLE_0   ;; This gets an address.
   mov ebp,[rbp]
   lea esi,[rbp+1]
   mov rdi,COUNT_TABLE_1   ;; This gets another address.
   mov [rdi],esi
   cmp ebp,64h
   je near .L34
```

This caused LCQ functions to not tier up when they're loaded from the
PTC cache. This does not happen when they're freshly compiled.

This PR fixes the issue by ensuring only a single counter is created per
translation.
2021-09-29 00:28:34 +02:00
FICTURE7
a9343c9364
Refactor PtcInfo (#2625)
* Refactor `PtcInfo`

This change reduces the coupling of `PtcInfo` by moving relocation
tracking to the backend. `RelocEntry`s remains as `RelocEntry`s through
out the pipeline until it actually needs to be written to the PTC
streams. Keeping this representation makes inspecting and manipulating
relocations after compilations less painful. This is something I needed
to do to patch relocations to 0 to diff dumps.

Contributes to #1125.

* Turn `Symbol` & `RelocInfo` into readonly structs

* Add documentation to `CompiledFunction`

* Remove `Compiler.Compile<T>`

Remove `Compiler.Compile<T>` and replace it by `Map<T>` of the
`CompiledFunction` returned.
2021-09-14 01:23:37 +02:00
FICTURE7
f2a7b300c4
Fix type mismatch in BitwiseAnd simplification (#2571)
* Fix type mismatch in `BitwiseAnd` simplification

`TryEliminateBitwiseAnd` would turn the `BitwiseAnd` operation into a
copy of the wrong type. E.g:

Before `Simplification`:
```llvm
i64 %0 = BitwiseAnd i64 0x0, %1
```

After `Simplication`:
```llvm
i64 %0 = Copy i32 0x0
```

Since the with the changes in #2515, we iterate in reverse order and
`Simplication`, `ConstantFolding` does not indicate if it modified
the CFG, the second pass to "retype" the copy into the proper
destination type does not happen.

This also blocked copy propagation since its destination type did not
match with its source type. But in the cases I've seen, the
`PreAllocator` would insert a copy for the propagated constant, which
results in no diffs.

Since the copy remained as is, asserts are fired when generating it.

* Set PPTC version
2021-08-20 14:42:00 -03:00
FICTURE7
22b2cb39af
Reduce JIT GC allocations (#2515)
* Turn `MemoryOperand` into a struct

* Remove `IntrinsicOperation`

* Remove `PhiNode`

* Remove `Node`

* Turn `Operand` into a struct

* Turn `Operation` into a struct

* Clean up pool management methods

* Add `Arena` allocator

* Move `OperationHelper` to `Operation.Factory`

* Move `OperandHelper` to `Operand.Factory`

* Optimize `Operation` a bit

* Fix `Arena` initialization

* Rename `NativeList<T>` to `ArenaList<T>`

* Reduce `Operand` size from 88 to 56 bytes

* Reduce `Operation` size from 56 to 40 bytes

* Add optimistic interning of Register & Constant operands

* Optimize `RegisterUsage` pass a bit

* Optimize `RemoveUnusedNodes` pass a bit

Iterating in reverse-order allows killing dependency chains in a single
pass.

* Fix PPTC symbols

* Optimize `BasicBlock` a bit

Reduce allocations from `_successor` & `DominanceFrontiers`

* Fix `Operation` resize

* Make `Arena` expandable

Change the arena allocator to be expandable by allocating in pages, with
some of them being pooled. Currently 32 pages are pooled. An LRU removal
mechanism should probably be added to it.

Apparently MHR can allocate bitmaps large enough to exceed the 16MB
limit for the type.

* Move `Arena` & `ArenaList` to `Common`

* Remove `ThreadStaticPool` & co

* Add `PhiOperation`

* Reduce `Operand` size from 56 from 48 bytes

* Add linear-probing to `Operand` intern table

* Optimize `HybridAllocator` a bit

* Add `Allocators` class

* Tune `ArenaAllocator` sizes

* Add page removal mechanism to `ArenaAllocator`

Remove pages which have not been used for more than 5s after each reset.

I am on fence if this would be better using a Gen2 callback object like
the one in System.Buffers.ArrayPool<T>, to trim the pool. Because right
now if a large translation happens, the pages will be freed only after a
reset. This reset may not happen for a while because no new translation
is hit, but the arena base sizes are rather small.

* Fix `OOM` when allocating larger than page size in `ArenaAllocator`

Tweak resizing mechanism for Operand.Uses and Assignemnts.

* Optimize `Optimizer` a bit

* Optimize `Operand.Add<T>/Remove<T>` a bit

* Clean up `PreAllocator`

* Fix phi insertion order

Reduce codegen diffs.

* Fix code alignment

* Use new heuristics for degree of parallelism

* Suppress warnings

* Address gdkchan's feedback

Renamed `GetValue()` to `GetValueUnsafe()` to make it more clear that
`Operand.Value` should usually not be modified directly.

* Add fast path to `ArenaAllocator`

* Assembly for `ArenaAllocator.Allocate(ulong)`:

  .L0:
    mov rax, [rcx+0x18]
    lea r8, [rax+rdx]
    cmp r8, [rcx+0x10]
    ja short .L2
  .L1:
    mov rdx, [rcx+8]
    add rax, [rdx+8]
    mov [rcx+0x18], r8
    ret
  .L2:
    jmp ArenaAllocator.AllocateSlow(UInt64)

  A few variable/field had to be changed to ulong so that RyuJIT avoids
  emitting zero-extends.

* Implement a new heuristic to free pooled pages.

  If an arena is used often, it is more likely that its pages will be
  needed, so the pages are kept for longer (e.g: during PPTC rebuild or
  burst sof compilations). If is not used often, then it is more likely
  that its pages will not be needed (e.g: after PPTC rebuild or bursts
  of compilations).

* Address riperiperi's feedback

* Use `EqualityComparer<T>` in `IntrusiveList<T>`

Avoids a potential GC hole in `Equals(T, T)`.
2021-08-17 15:08:34 -03:00
FICTURE7
9d7627af64
Add multi-level function table (#2228)
* Add AddressTable<T>

* Use AddressTable<T> for dispatch

* Remove JumpTable & co.

* Add fallback for out of range addresses

* Add PPTC support

* Add documentation to `AddressTable<T>`

* Make AddressTable<T> configurable

* Fix table walk

* Fix IsMapped check

* Remove CountTableCapacity

* Add PPTC support for fast path

* Rename IsMapped to IsValid

* Remove stale comment

* Change format of address in exception message

* Add TranslatorStubs

* Split DispatchStub

Avoids recompilation of stubs during tests.

* Add hint for 64bit or 32bit

* Add documentation to `Symbol`

* Add documentation to `TranslatorStubs`

Make `TranslatorStubs` disposable as well.

* Add documentation to `SymbolType`

* Add `AddressTableEventSource` to monitor function table size

Add an EventSource which measures the amount of unmanaged bytes
allocated by AddressTable<T> instances.

 dotnet-counters monitor -n Ryujinx --counters ARMeilleure

* Add `AllowLcqInFunctionTable` optimization toggle

This is to reduce the impact this change has on the test duration.
Before everytime a test was ran, the FunctionTable would be initialized
and populated so that the newly compiled test would get registered to
it.

* Implement unmanaged dispatcher

Uses the DispatchStub to dispatch into the next translation, which
allows execution to stay in unmanaged for longer and skips a
ConcurrentDictionary look up when the target translation has been
registered to the FunctionTable.

* Remove redundant null check

* Tune levels of FunctionTable

Uses 5 levels instead of 4 and change unit of AddressTableEventSource
from KB to MB.

* Use 64-bit function table

Improves codegen for direct branches:

    mov qword [rax+0x408],0x10603560
 -  mov rcx,sub_10603560_OFFSET
 -  mov ecx,[rcx]
 -  mov ecx,ecx
 -  mov rdx,JIT_CACHE_BASE
 -  add rdx,rcx
 +  mov rcx,sub_10603560
 +  mov rdx,[rcx]
    mov rcx,rax

Improves codegen for dispatch stub:

    and rax,byte +0x1f
 -  mov eax,[rcx+rax*4]
 -  mov eax,eax
 -  mov rcx,JIT_CACHE_BASE
 -  lea rax,[rcx+rax]
 +  mov rax,[rcx+rax*8]
    mov rcx,rbx

* Remove `JitCacheSymbol` & `JitCache.Offset`

* Turn `Translator.Translate` into an instance method

We do not have to add more parameter to this method and related ones as
new structures are added & needed for translation.

* Add symbol only when PTC is enabled

Address LDj3SNuD's feedback

* Change `NativeContext.Running` to a 32-bit integer

* Fix PageTable symbol for host mapped
2021-05-29 18:06:28 -03:00
riperiperi
54ea2285f0
POWER - Performance Optimizations With Extensive Ramifications (#2286)
* Refactoring of KMemoryManager class

* Replace some trivial uses of DRAM address with VA

* Get rid of GetDramAddressFromVa

* Abstracting more operations on derived page table class

* Run auto-format on KPageTableBase

* Managed to make TryConvertVaToPa private, few uses remains now

* Implement guest physical pages ref counting, remove manual freeing

* Make DoMmuOperation private and call new abstract methods only from the base class

* Pass pages count rather than size on Map/UnmapMemory

* Change memory managers to take host pointers

* Fix a guest memory leak and simplify KPageTable

* Expose new methods for host range query and mapping

* Some refactoring of MapPagesFromClientProcess to allow proper page ref counting and mapping without KPageLists

* Remove more uses of AddVaRangeToPageList, now only one remains (shared memory page checking)

* Add a SharedMemoryStorage class, will be useful for host mapping

* Sayonara AddVaRangeToPageList, you served us well

* Start to implement host memory mapping (WIP)

* Support memory tracking through host exception handling

* Fix some access violations from HLE service guest memory access and CPU

* Fix memory tracking

* Fix mapping list bugs, including a race and a error adding mapping ranges

* Simple page table for memory tracking

* Simple "volatile" region handle mode

* Update UBOs directly (experimental, rough)

* Fix the overlap check

* Only set non-modified buffers as volatile

* Fix some memory tracking issues

* Fix possible race in MapBufferFromClientProcess (block list updates were not locked)

* Write uniform update to memory immediately, only defer the buffer set.

* Fix some memory tracking issues

* Pass correct pages count on shared memory unmap

* Armeilleure Signal Handler v1 + Unix changes

Unix currently behaves like windows, rather than remapping physical

* Actually check if the host platform is unix

* Fix decommit on linux.

* Implement windows 10 placeholder shared memory, fix a buffer issue.

* Make PTC version something that will never match with master

* Remove testing variable for block count

* Add reference count for memory manager, fix dispose

Can still deadlock with OpenAL

* Add address validation, use page table for mapped check, add docs

Might clean up the page table traversing routines.

* Implement batched mapping/tracking.

* Move documentation, fix tests.

* Cleanup uniform buffer update stuff.

* Remove unnecessary assignment.

* Add unsafe host mapped memory switch

On by default. Would be good to turn this off for untrusted code (homebrew, exefs mods) and give the user the option to turn it on manually, though that requires some UI work.

* Remove C# exception handlers

They have issues due to current .NET limitations, so the meilleure one fully replaces them for now.

* Fix MapPhysicalMemory on the software MemoryManager.

* Null check for GetHostAddress, docs

* Add configuration for setting memory manager mode (not in UI yet)

* Add config to UI

* Fix type mismatch on Unix signal handler code emit

* Fix 6GB DRAM mode.

The size can be greater than `uint.MaxValue` when the DRAM is >4GB.

* Address some feedback.

* More detailed error if backing memory cannot be mapped.

* SetLastError on all OS functions for consistency

* Force pages dirty with UBO update instead of setting them directly.

Seems to be much faster across a few games. Need retesting.

* Rebase, configuration rework, fix mem tracking regression

* Fix race in FreePages

* Set memory managers null after decrementing ref count

* Remove readonly keyword, as this is now modified.

* Use a local variable for the signal handler rather than a register.

* Fix bug with buffer resize, and index/uniform buffer binding.

Should fix flickering in games.

* Add InvalidAccessHandler to MemoryTracking

Doesn't do anything yet

* Call invalid access handler on unmapped read/write.

Same rules as the regular memory manager.

* Make unsafe mapped memory its own MemoryManagerType

* Move FlushUboDirty into UpdateState.

* Buffer dirty cache, rather than ubo cache

Much cleaner, may be reusable for Inline2Memory updates.

* This doesn't return anything anymore.

* Add sigaction remove methods, correct a few function signatures.

* Return empty list of physical regions for size 0.

* Also on AddressSpaceManager

Co-authored-by: gdkchan <gab.dark.100@gmail.com>
2021-05-24 22:52:44 +02:00
gdkchan
fb65f392d1
Improve accuracy of reciprocal step instructions (#2305)
* Improve accuracy of reciprocal step instructions

* Fix small mistake on RECPE rounding, nits, PTC version bump
2021-05-24 20:20:07 +10:00
FICTURE7
65ac00833a
Use branch instead of tailcall for recursive calls (#2282)
* Use branch instead of tailcall for recursive calls

Use a branch instead of doing a tailcall for recursive calls. This
avoids having to store the dispatch address, setting up the epilogue and
keeps guest registers in host registers for longer.

The rejit check is moved down into the entry block so that the rejit
behaviour remains the same as before.

* Set PTC version

Co-authored-by: gdkchan <gab.dark.100@gmail.com>
2021-05-20 09:31:45 -03:00
FICTURE7
0181068016
Add BIC/ORR Vd.T, #imm fast path (#2279)
* Add fast path for BIC Vd.T, #imm

* Add fast path for ORR Vd.T, #imm

* Set PTC version

* Fixup Exception to InvalidOperationException
2021-05-20 09:09:17 -03:00
FICTURE7
c805542b29
Allow LocalVariable to be assigned more than once (#2288)
* Allow `LocalVariable` to be assigned more than once

This allows us to write flow controls like loops and if-elses with
LocalVariables participating in phi nodes.

* Add `GetLocalNumber` to operand
2021-05-17 01:54:53 +02:00
gdkchan
e318022b89
Fold constant offsets and group constant addresses (#2285)
* Fold constant offsets and group constant addresses

* PPTC version bump
2021-05-13 21:26:57 +02:00
LDj3SNuD
57ea3f93a3
PPTC meets ExeFS Patching. (#1865)
* PPTC meets ExeFS Patching.

* InternalVersion = 1865

* Ready!

* Optimized the PtcProfiler Load/Save methods.
2021-05-13 20:05:15 +02:00
FICTURE7
89791ba68d
Add inlined on translation call counting (#2190)
* Add EntryTable<TEntry>

* Add on translation call counting

* Add Counter

* Add PPTC support

* Make Counter a generic & use a 32-bit counter instead

* Return false on overflow

* Set PPTC version

* Print more information about the rejit queue

* Make Counter<T> disposable

* Remove Block.TailCall since it is not used anymore

* Apply suggestions from code review

Address gdkchan's feedback

Co-authored-by: gdkchan <gab.dark.100@gmail.com>

* Fix more stale docs

* Remove rejit requests queue logging

* Make Counter<T> finalizable

Most certainly quite an odd use case.

* Make EntryTable<T>.TryAllocate set entry to default

* Re-trigger CI

* Dispose Counters before they hit the finalizer queue

* Re-trigger CI

Just for good measure...

* Make EntryTable<T> expandable

* EntryTable is now expandable instead of being a fixed slab.
* Remove EntryTable<T>.TryAllocate
* Remove Counter<T>.TryCreate

Address LDj3SNuD's feedback

* Apply suggestions from code review

Address LDj3SNuD's feedback

Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>

* Remove useless return

* POH approach, but the sequel

* Revert "POH approach, but the sequel"

This reverts commit 5f5abaa247.

The sequel got shelved

* Add extra documentation

Co-authored-by: gdkchan <gab.dark.100@gmail.com>
Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>
2021-04-18 23:43:53 +02:00
LDj3SNuD
90163087a0
PPTC vs. giant ExeFS. (#2168)
* PPTC vs. giant ExeFS.

* InternalVersion = 2168

* Add new heuristic algorithm for calculating the number of threads for parallel translations that also takes into account the user's free physical memory and not just the number of CPU cores.

* Nit.

* Add an outer Header structure and add the hashes for both this new structure and the existing "inner" Header structure.

* InternalVersion = 2169
2021-04-13 03:24:36 +02:00
FICTURE7
98ed81e4cd
Improve StoreToContext emission (#2155)
* Improve StoreToContext emission

Hoist StoreToContext in dynamic branch fast & slow paths out into
their predecessor.

Reduces register pressure, code size and compile time because we're
throwing less stuff down the pipeline.

* Set PTC internal version

* Turn EmitDynamicTableCall private

* Re-trigger CI
2021-04-02 19:54:23 +02:00
FICTURE7
8b3eba7e13
Reduce allocation during SSA construction (#2162)
* Reduce allocation during SSA construction

* Re-trigger CI
2021-04-02 19:26:16 +02:00
mageven
69f8722e79
Fix inconsistencies in progress reporting (#2129)
Signal and setup events correctly
Eliminate possible races
Use a single event
Mark volatiles and reduce scope of waithandles
Common handler
100ms -> 50ms
2021-03-22 19:40:07 +01:00
mageven
ca5d8e58dd
Add progress reporting to PTC and Shader Cache (#2057)
* UI changes

* Add progress reporting to PTC & ShaderCache

* Account for null events and expand docs

Co-authored-by: Joshi234 <46032261+Joshi234@users.noreply.github.com>
2021-03-03 01:39:36 +01:00
LDj3SNuD
bcbf240d2e
PPTC: Fix unwanted propagation of a relocatable constant in a specific case. (#1990)
* Fix unwanted propagation of a relocatable constant in a specific case.

* Ptc.InternalVersion = 1990

* Nit to retrigger the Checks.
2021-02-23 13:15:45 +01:00
mageven
9bda7b4699
Implement VCNT instruction (#1963)
* Implement VCNT based on AArch64 CNT

Add tests

* Update PTC version

* Address LDj's comments

* Explicit size in encoding
* Tighter tests
* Replace SoftFallback with IR helper

Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>

* Reduce one BitwiseAnd from IR fallback

Based on popcount64b from https://en.wikipedia.org/wiki/Hamming_weight#Efficient_implementation

* Rename parameter and add assert

Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>

Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>
2021-02-22 16:26:13 +01:00
LDj3SNuD
dc0adb533d
PPTC & Pool Enhancements. (#1968)
* PPTC & Pool Enhancements.

* Avoid buffer allocations in CodeGenContext.GetCode(). Avoid stream allocations in PTC.PtcInfo.

Refactoring/nits.

* Use XXHash128, for Ptc.Load & Ptc.Save, x10 faster than Md5.

* Why not a nice Span.

* Added a simple PtcFormatter library for deserialization/serialization, which does not require reflection, in use at PtcJumpTable and PtcProfiler; improves maintainability and simplicity/readability of affected code.

* Nits.

* Revert #1987.

* Revert "Revert #1987."

This reverts commit 998be765cf.
2021-02-22 03:23:48 +01:00
FICTURE7
1586880114
Turn Copy into Fill in HybridAllocator (#2010)
* Turn Copy into Fill in HybridAllocator

* Set PTC internal verison
2021-02-21 18:33:59 +01:00
gdkchan
9d82d27df2
Fix memory tracking performance regression (#2026)
* Fix memory tracking performance regression

* Set PTC version
2021-02-17 09:16:20 +11:00
gdkchan
715b605e95
Validate CPU virtual addresses on access (#1987)
* Enable PTE null checks again

* Do address validation on EmitPtPointerLoad, and make it branchless

* PTC version increment

* Mask of pointer tag for exclusive access

* Move mask to the correct place

Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>
2021-02-16 19:04:19 +01:00
sharmander
40797a1283
Optimization | Modify Add (Integer) Instruction to use LEA instead. (#1971)
* Optimization | Modify Add Instruction to use LEA instead.

Currently, the add instruction requires 4 registers to take place. By using LEA, we can effectively perform the same working using 3 registers, reducing memory spills and improving translation efficiency.

* Fix IsSameOperandDestSrc1 Check for Add

* Use LEA if Dest != SRC1

* Update IsSameOperandDestSrc1 to account for Cases where Dest and Src1 can be same for add

* Fix error in logic

* Typo

* Add paranthesis for clarity

* Compare registers as requested.

* Cleanup if statement, use same comparison method as generateCopy

* Make change as recommended by gdk

* Perform check only when Add calls are made

* use ensure sametype for lea, fix else

* Update comment

* Update version #
2021-02-08 10:49:46 +11:00
gdkchan
ee28ccebf4
Disable partial JIT invalidation on unmap (#1991) 2021-02-08 10:25:14 +11:00
gdkchan
dcce407071
Lower precision of estimate instruction results to match Arm behavior (#1943)
* Lower precision of estimate instruction results to match Arm behavior

* PTC version update

* Nits
2021-01-28 10:23:00 +11:00
mageven
c19cfca183
Implement PRFM (register variant) as NOP (#1956)
* Implement PRFM (register variant) as NOP

Fix typo pfrm -> prfm
Add comments to distinguish variants

* Increment PTC version
2021-01-26 16:09:27 +11:00
FICTURE7
ddf1105bcb
Add VCLZ.* fast path (#1917)
* Add VCLZ fast path

* Add VCLZ.8B/16B SSSE3 fast path
* Add VCLZ.4H/8H SSSE3 fast path
* Add VCLZ.2S/4S SSE2 fast path

* Improve CLZ.4H/8H fast path

* Improve CLZ.2S/4S fast path

* Set PPTC version
2021-01-25 10:01:25 +11:00
LDj3SNuD
68f6b79fd3
Add a simple Pools Limiter. (#1830)
* Added support for offline invalidation, via PPTC, of low cq translations replaced by high cq translations; both on a single run and between runs.

Added invalidation of .cache files in the event of reuse on a different user operating system.

Added .info and .cache files invalidation in case of a failed stream decompression.

Nits.

* InternalVersion = 1712;

* Nits.

* Address comment.

* Get rid of BinaryFormatter.

Nits.

* Move Ptc.LoadTranslations().

Nits.

* Nits.

* Fixed corner cases (in case backup copies have to be used). Added save logs.

* Not core fixes.

* Complement to the previous commit. Added load logs. Removed BinaryFormatter leftovers.

* Add LoadTranslations log.

* Nits.

* Removed the search and management of LowCq overlapping functions.

* Final increment of .info and .cache flags.

* Nit.

* Free up memory allocated by Pools during any PPTC translations at boot time.

* Nit due to rebase.

* Add a simple Pools Limiter.

* Nits.

* Fix missing JumpTable.RegisterFunction() due to rebase.

Clear MemoryStreams as soon as possible, when they are no longer needed.

* Code cleaning.

* Nit for retrigger Checks.

* Update Ptc.cs

* Contextual refactoring of Translator. Ignore resetting of pools for DirectCallStubs.

* Nit for retrigger Checks.
2021-01-12 19:04:02 +01:00
LDj3SNuD
430ba6da65
CPU (A64): Add Pmull_V Inst. with Clmul fast path for the "1/2D -> 1Q" variant & Sse fast path and slow path for both the "8/16B -> 8H" and "1/2D -> 1Q" variants; with Test. (#1817)
* Add Pmull_V Sse fast path only, both "8/16B -> 8H" and "1/2D -> 1Q" variants; with Test.

* Add Clmul fast path for the 128 bits variant.

* Small optimisation (save 60 instructions) for the Sse fast path about the 128 bits variant.

* Add slow path, both variants. Fix V128 Shl/Shr when shift = 0.

* A32: Add Vmull_I P64 variant (slow path); not tested.

* A32: Add Vmull_I_P8_P64 Test and fix P64 variant.
2021-01-04 23:45:54 +01:00
Ac_K
5b9c876155 Hotfix for #1814 2020-12-24 04:44:39 +01:00
LDj3SNuD
2502f1f07f
Free up memory allocated by Pools during any PPTC translations at boot time. (#1814)
* Added support for offline invalidation, via PPTC, of low cq translations replaced by high cq translations; both on a single run and between runs.

Added invalidation of .cache files in the event of reuse on a different user operating system.

Added .info and .cache files invalidation in case of a failed stream decompression.

Nits.

* InternalVersion = 1712;

* Nits.

* Address comment.

* Get rid of BinaryFormatter.

Nits.

* Move Ptc.LoadTranslations().

Nits.

* Nits.

* Fixed corner cases (in case backup copies have to be used). Added save logs.

* Not core fixes.

* Complement to the previous commit. Added load logs. Removed BinaryFormatter leftovers.

* Add LoadTranslations log.

* Nits.

* Removed the search and management of LowCq overlapping functions.

* Final increment of .info and .cache flags.

* Nit.

* Free up memory allocated by Pools during any PPTC translations at boot time.

* Nit due to rebase.
2020-12-24 03:58:36 +01:00
LDj3SNuD
8a33e884f8
Fix Vnmls_S fast path (F64: losing input d value). Fix Vnmla_S & Vnmls_S slow paths (using fused inst.s). Fix Vfma_V slow path not using StandardFPSCRValue(). (#1775)
* Fix Vnmls_S fast path (F64: losing input d value). Fix Vnmla_S & Vnmls_S slow paths (using fused inst.s).

Add Vfma_S & Vfms_S Fma fast paths.
Add Vfnma_S inst. with Fma/Sse fast paths and slow path.
Add Vfnms_S Sse fast path.

Add Tests for affected inst.s.

Nits.

* InternalVersion = 1775

* Nits.

* Fix Vfma_V slow path not using StandardFPSCRValue().

* Nit: Fix Vfma_V order.

* Add Vfms_V Sse fast path and slow path.

* Add Vfma_V and Vfms_V Test.
2020-12-17 20:43:41 +01:00
LDj3SNuD
b5c215111d
PPTC Follow-up. (#1712)
* Added support for offline invalidation, via PPTC, of low cq translations replaced by high cq translations; both on a single run and between runs.

Added invalidation of .cache files in the event of reuse on a different user operating system.

Added .info and .cache files invalidation in case of a failed stream decompression.

Nits.

* InternalVersion = 1712;

* Nits.

* Address comment.

* Get rid of BinaryFormatter.

Nits.

* Move Ptc.LoadTranslations().

Nits.

* Nits.

* Fixed corner cases (in case backup copies have to be used). Added save logs.

* Not core fixes.

* Complement to the previous commit. Added load logs. Removed BinaryFormatter leftovers.

* Add LoadTranslations log.

* Nits.

* Removed the search and management of LowCq overlapping functions.

* Final increment of .info and .cache flags.

* Nit.

* GetIndirectFunctionAddress(): Validate that writing actually takes place in dynamic table memory range (and not elsewhere).

* Fix Ptc.UpdateInfo() due to rebase.

* Nit for retrigger Checks.

* Nit for retrigger Checks.
2020-12-17 20:32:09 +01:00
gdkchan
61634dd415
Clear JIT cache on exit (#1518)
* Initial cache memory allocator implementation

* Get rid of CallFlag

* Perform cache cleanup on exit

* Basic cache invalidation

* Thats not how conditionals works in C# it seems

* Set PTC version to PR number

* Address PR feedback

* Update InstEmitFlowHelper.cs

* Flag clear on address is no longer needed

* Do not include exit block in function size calculation

* Dispose jump table

* For future use

* InternalVersion = 1519 (force retest).

Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>
2020-12-16 17:07:42 -03:00
gdkchan
c8bb3cc50e
Fix register read after write on STREX implementation (#1801)
* Fix register read after write on STREX implementation

* PTC version update
2020-12-13 12:19:38 -03:00
sharmander
36f6bbf5b9
CPU: Implement VFNMA.F32 | F.64 (#1783)
* Implement VFNMA.F<32/64>

* Update PTC Version

* Update Implementation & Renames & Correct Order

* Fix alignment

* Update implementation to not trigger assert

* Actually use the intrinsic that makes sense :)
2020-12-07 21:04:01 -03:00
LDj3SNuD
567ea726e1
Add support for guest Fz (Fpcr) mode through host Ftz and Daz (Mxcsr) modes (fast paths). (#1630)
* Add support for guest Fz (Fpcr) mode through host Ftz and Daz (Mxcsr) modes (fast paths).

* Ptc.InternalVersion = 1630

* Nits.

* Address comments.

* Update Ptc.cs

* Address comment.
2020-12-07 10:37:07 +01:00
sharmander
b479a43939
CPU: Implement VFNMS.F32/64 (#1758)
* Add necessary methods / op-code

* Enable Support for FMA Instruction Set

* Add Intrinsics / Assembly Opcodes for VFMSUB231XX.

* Add X86 Instructions for VFMSUB231XX

* Implement VFNMS

* Implement VFNMS Tests

* Add special cases for FMA instructions.

* Update PPTC Version

* Remove unused Op

* Move Check into Assert / Cleanup

* Rename and cleanup

* Whitespace

* Whitespace / Rename

* Re-sort

* Address final requests

* Implement VFMA.F64

* Simplify switch

* Simplify FMA Instructions into their own IntrinsicType.

* Remove whitespace

* Fix indentation

* Change tests for Vfnms -- disable inf / nan

* Move args up, not description ;)

* Undo vfma

* Completely remove vfms code.,

* Fix order of instruction in assembler
2020-12-03 20:20:02 +01:00
riperiperi
9852cb9c9e
Use backup when PTC compression is corrupt (#1704)
* Use backup when PTC compression is corrupt

* Apply suggestions from code review

Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>

Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>
2020-11-20 02:51:59 +01:00
LDj3SNuD
0679084f11
CPU (A64): Add FP16/FP32 fast paths (F16C Intrinsics) for Fcvt_S, Fcvtl_V & Fcvtn_V Instructions. Now HardwareCapabilities uses CpuId. (#1650)
* net5.0

* CPU (A64): Add FP16/FP32 fast paths (F16C Intrinsics) for Fcvt_S, Fcvtl_V & Fcvtn_V Instructions. Switch to .NET 5.0.

Nits.

Tests performed successfully in both debug and release mode (for all instructions involved).

* Address comment.

* Update appveyor.yml

* Revert "Update appveyor.yml"

This reverts commit 27cdd59e8b.

* Remove Assembler CpuId.

* Update appveyor.yml

* Address comment.
2020-11-18 19:35:54 +01:00
Mary
863edae328
shader cache: Fix Linux boot issues (#1709)
* shader cache: Fix Linux boot issues

This rollback the init logic back to previous state, and replicate the
way PTC handle initialization.

* shader cache: set default state of ready for translation event to false

* Fix cpu unit tests
2020-11-17 22:40:19 +01:00
riperiperi
b4d8d893a4
Memory Read/Write Tracking using Region Handles (#1272)
* WIP Range Tracking

- Texture invalidation seems to have large problems
- Buffer/Pool invalidation may have problems
- Mirror memory tracking puts an additional `add` in compiled code, we likely just want to make HLE access slower if this is the final solution.
- Native project is in the messiest possible location.
- [HACK] JIT memory access always uses native "fast" path
- [HACK] Trying some things with texture invalidation and views.

It works :)

Still a few hacks, messy things, slow things

More work in progress stuff (also move to memory project)

Quite a bit faster now.
- Unmapping GPU VA and CPU VA will now correctly update write tracking regions, and invalidate textures for the former.
- The Virtual range list is now non-overlapping like the physical one.
- Fixed some bugs where regions could leak.
- Introduced a weird bug that I still need to track down (consistent invalid buffer in MK8 ribbon road)

Move some stuff.

I think we'll eventually just put the dll and so for this in a nuget package.

Fix rebase.

[WIP] MultiRegionHandle variable size ranges

- Avoid reprotecting regions that change often (needs some tweaking)
- There's still a bug in buffers, somehow.
- Might want different api for minimum granularity

Fix rebase issue

Commit everything needed for software only tracking.

Remove native components.

Remove more native stuff.

Cleanup

Use a separate window for the background context, update opentk. (fixes linux)

Some experimental changes

Should get things working up to scratch - still need to try some things with flush/modification and res scale.

Include address with the region action.

Initial work to make range tracking work

Still a ton of bugs

Fix some issues with the new stuff.

* Fix texture flush instability

There's still some weird behaviour, but it's much improved without this. (textures with cpu modified data were flushing over it)

* Find the destination texture for Buffer->Texture full copy

Greatly improves performance for nvdec videos (with range tracking)

* Further improve texture tracking

* Disable Memory Tracking for view parents

This is a temporary approach to better match behaviour on master (where invalidations would be soaked up by views, rather than trigger twice)

The assumption is that when views are created to a texture, they will cover all of its data anyways. Of course, this can easily be improved in future.

* Introduce some tracking tests.

WIP

* Complete base tests.

* Add more tests for multiregion, fix existing test.

* Cleanup Part 1

* Remove unnecessary code from memory tracking

* Fix some inconsistencies with 3D texture rule.

* Add dispose tests.

* Use a background thread for the background context.

Rather than setting and unsetting a context as current, doing the work on a dedicated thread with signals seems to be a bit faster.

Also nerf the multithreading test a bit.

* Copy to texture with matching alignment

This extends the copy to work for some videos with unusual size, such as tutorial videos in SMO. It will only occur if the destination texture already exists at XCount size.

* Track reads for buffer copies. Synchronize new buffers before copying overlaps.

* Remove old texture flushing mechanisms.

Range tracking all the way, baby.

* Wake the background thread when disposing.

Avoids a deadlock when games are closed.

* Address Feedback 1

* Separate TextureCopy instance for background thread

Also `BackgroundContextWorker.InBackground` for a more sensible idenfifier for if we're in a background thread.

* Add missing XML docs.

* Address Feedback

* Maybe I should start drinking coffee.

* Some more feedback.

* Remove flush warning, Refocus window after making background context
2020-10-16 17:18:35 -03:00
LDj3SNuD
04e330cc00
Add Umaal & Vabd_I, Vabdl_I, Vaddl_I, Vhadd, Vqshrn, Vshll inst.s (slow paths). (#1577)
* Add Umaal & Vabd_I, Vabdl_I, Vaddl_I, Vhadd, Vqshrn, Vshll inst.s (slow paths).

No test provided (i.e. draft).

* Ptc InternalVersion = 1577
2020-10-13 22:41:33 +02:00
gdkchan
f2b12c9749
Remove old, unused CPU optimization (#1586) 2020-09-30 16:16:34 -03:00
gdkchan
6c9565693f
IPC refactor part 1: Use explicit separate threads to process requests (#1447)
* Changes to allow explicit management of service threads

* Remove now unused code

* Remove ThreadCounter, its no longer needed

* Allow and use separate server per service, also fix exit issues

* New policy change: PTC version now uses PR number
2020-09-22 14:50:40 +10:00
LDj3SNuD
66b799a6e4
Fix host stack overflow caused by some recursive guest methods. (#1528)
* Fix host stack overflow caused by some recursive guest methods.

* PPTC flag up.

* Address comments.

Co-authored-by: gdkchan <gab.dark.100@gmail.com>
2020-09-19 20:16:30 -03:00
FICTURE7
f60033e0aa
Implement block placement (#1549)
* Implement block placement

Implement a simple pass which re-orders cold blocks at the end of the
list of blocks in the CFG.

* Set PPTC version

* Use Array.Resize

Address gdkchan's feedback
2020-09-19 20:00:24 -03:00
FICTURE7
36ec1bc6c0
Relax block ordering constraints (#1535)
* Relax block ordering constraints

Before `block.Next` had to follow `block.ListNext`, now it does not.
Instead `CodeGenerator` will now emit the necessary jump instructions
to ensure control flow.

This makes control flow and block order modifications easier. It also
eliminates some simple cases of redundant branches.

* Set PPTC version
2020-09-12 12:32:53 -03:00
FICTURE7
4c7bebf3e6
Do not emit StoreToContext before Return (#1537)
* Do not emit StoreToContext before Return

* Set PPTC version
2020-09-07 12:52:17 +10:00
gdkchan
6cc187da59
SIMD&FP load/store with scale > 4 should be undefined (#1522)
* SIMD&FP load/store with scale > 4 should be undefined

* Catch more invalid encodings for FP&SIMD LDR/STR (reg variant)

* Set PTC version to PR number
2020-09-01 17:02:23 -03:00
FICTURE7
92f7f163ef
Improve static branch prediction along fast path for memory accesses (#1484)
* Improve static branch prediction along fast path for memory accesses

* Set PPTC interval version
2020-08-31 20:55:15 -03:00
mageven
b9398f1f3a
Allow launching with custom data directories (#1505)
* Allow launching with custom data directories

Don't load alternate keys when using custom directory

* Address gdkchan's comments

* Misc fixes to log levels

Added more enabled log levels by default

Moved successful config updation to Notice as
1. It's not a warning
2. Warnings could've been disabled by the config load and hence message
   would be lost
2020-08-30 18:51:53 +02:00
LDj3SNuD
6938988427
Fix Vcvt_FI & Vcvt_RM; Add Vfma_S & Vfms_S. Add Tests. (#1471)
* Fix Vcvt_FI & Vcvt_RM; Add Vfma_S & Vfms_S. Add Tests.

* Address PR feedback & Nit.
2020-08-13 02:34:02 -03:00
mageven
9c6a3aacc2
Fix PTC version increment from #1433 (#1462) 2020-08-09 05:32:00 +02:00
LDj3SNuD
e36e97c64d
CPU: This PR fixes Fpscr, among other things. (#1433)
* CPU: This PR fixes Fpscr, among other things.

* Add Fpscr.Qc = 1 if sat. for Vqrshrn & Vqrshrun.

* Fix Vcmp & Vcmpe opcode table.

* Revert "Fix Vcmp & Vcmpe opcode table."

This reverts commit c117d9410d.

* Address PR feedbacks.
2020-08-08 17:18:51 +02:00
Ficture Seven
ee22517d92
Improve branch operations (#1442)
* Add Compare instruction

* Add BranchIf instruction

* Use test when BranchIf & Compare against 0

* Propagate Compare into BranchIfTrue/False use

- Propagate Compare operations into their BranchIfTrue/False use and
  turn these into a BranchIf.

- Clean up Comparison enum.

* Replace BranchIfTrue/False with BranchIf

* Use BranchIf in EmitPtPointerLoad

- Using BranchIf early instead of BranchIfTrue/False improves LCQ and
  reduces the amount of work needed by the Optimizer.

  EmitPtPointerLoader was a/the big producer of BranchIfTrue/False.

- Fix asserts firing when assembling BitwiseAnd because of type
  mismatch in EmitStoreExclusive. This is harmless and should not
  cause any diffs.

* Increment PPTC interval version

* Improve IRDumper for BranchIf & Compare

* Use BranchIf in EmitNativeCall

* Clean up

* Do not emit test when immediately preceded by and
2020-08-05 08:52:33 +10:00
mageven
a33dc2f491
Improved Logger (#1292)
* Logger class changes only

Now compile-time checking is possible with the help of Nullable Value
types.

* Misc formatting

* Manual optimizations

PrintGuestLog
PrintGuestStackTrace
Surfaceflinger DequeueBuffer

* Reduce SendVibrationXX log level to Debug

* Add Notice log level

This level is always enabled and used to print system info, etc...
Also, rewrite LogColor to switch expression as colors are static

* Unify unhandled exception event handlers

* Print enabled LogLevels during init

* Re-add App Exit disposes in proper order

nit: switch case spacing

* Revert PrintGuestStackTrace to Info logs due to #1407

PrintGuestStackTrace is now called in some critical error handlers
so revert to old behavior as KThread isn't part of Guest.

* Batch replace Logger statements
2020-08-04 01:32:53 +02:00
gdkchan
9878fc2d3c
Implement inline memory load/store exclusive and ordered (#1413)
* Implement inline memory load/store exclusive

* Fix missing REX prefix on 8-bits CMPXCHG

* Increment PTC version due to bugfix

* Remove redundant memory checks

* Address PR feedback

* Increment PPTC version
2020-07-30 11:29:28 -03:00
Ficture Seven
b3c051bbec
Use movd,movq for i32/64 VectorExtract %x, 0x0 (#1439)
* Use movd,movq for i32/64 VectorExtract %x, 0x0

* Increment PPTC interval version

* Use else-if instead

- Address gdkchan's feedback.
- Clean up Debug.Assert calls

* Inline `count` expression into Debug.Assert

Apparently the CoreCLR JIT will not eliminate this. :(
2020-07-30 15:52:26 +10:00
gdkchan
2678bf0010
PPTC version increment (#1427) 2020-07-25 20:01:57 +02:00
Valentin PONS
3af2ce74ec
Implements some 32-bit instructions (VBIC, VTST, VSRA) (#1192)
* Added some 32 bits instructions:

* VBIC
* VTST
* VSRA

* Incremented the PTC

* Add tests and fix implementation

* Fixed VBIC immediate opcode mapping

* Hey hey!

* Nit.

Co-authored-by: gdkchan <gab.dark.100@gmail.com>
Co-authored-by: LDj3SNuD <dvitiello@gmail.com>
Co-authored-by: LDj3SNuD <35856442+LDj3SNuD@users.noreply.github.com>
2020-07-19 15:11:58 -03:00
LDj3SNuD
56a61a5758
CPU: A32: Fix Vabs_V & Vneg_V (S8, S16, S32 & F32); add Tests. (#1394)
* Fix Vabs_V & Vneg_V (S8, S16, S32 & F32); add Tests.

* Update Ptc.cs
2020-07-17 10:57:49 -03:00
LDj3SNuD
88619d71b8
CPU: A32: Add Vadd & Vsub Wide (S/U_8/16/32) Inst.s with Test. (#1390) 2020-07-17 14:21:40 +10:00
LDj3SNuD
a804db6eed
Add Fmax/minv_V & S/Ushl_S Inst.s with Tests. Fix Maxps/d & Minps/d d… (#1335)
* Add Fmax/minv_V & S/Ushl_S Inst.s with Tests. Fix Maxps/d & Minps/d double zero sign handling. Allows better handling of NaNs.

* Optimized EmitSse2VectorIsNaNOpF() for multiple uses per opF.
2020-07-13 21:08:47 +10:00
riperiperi
d7044b10a2
Add SSE4.2 Path for CRC32, add A32 variant, add tests for non-castagnoli variants. (#1328)
* Add CRC32 A32 instructions.

* Fix CRC32 instructions.

* Add CRC intrinsic and fast path.

Loop is currently unrolled, will look into adding temp vars after tests are added.

* Begin work on Crc tests

* Fix SSE4.2 path for CRC32C, finialize tests.

* Remove unused IR path.

* Fix spacing between prefix checks.

* This should be Src.

* PTC Version

* OpCodeTable Order

* Integer check improvement. Value and Crc can be either 32 or 64 size.

* This wasn't necessary...

* If size is 3, value type must be I64.

* Fix same src+dest handling for non crc intrinsics.

* Pre-fix (ha) issue with vex encodings
2020-07-13 20:48:14 +10:00
Ficture Seven
30d4f752f4
Fix Node Uses/Assignments (#1376)
* Fix Node Uses/Assignments

* Bump PPTC Version Number

Co-authored-by: jduncanator <1518948+jduncanator@users.noreply.github.com>
2020-07-13 20:20:07 +10:00
Ficture Seven
b5127131c5
Fix folding of ConvertI64ToI32 imm64 (#1383)
* Fix folding of ConvertI64ToI32 imm64

* Increment PTC internal version

* Clean up
2020-07-13 19:10:33 +10:00
gdkchan
38b26cf424
Mask shift constants on x86 backend (#1382)
* Mask shift constants on x86 backendd

* Version bump
2020-07-11 15:52:38 +10:00
Ficture Seven
0195d0938d
Fold ZeroExtend8/16/32 imm32/64 (#1358)
* Fold ZeroExtend8/16/32 imm32/64

* Increment PTC version
2020-07-11 11:00:41 +10:00
Ficture Seven
7639bb2e86
Fold ConvertI64ToI32 imm64 (#1359)
* Fold ConvertI64ToI32 imm64

* Increment PTC version

* Bump PPTC InternalVersion

Co-authored-by: jduncanator <1518948+jduncanator@users.noreply.github.com>
2020-07-11 10:35:38 +10:00
LDj3SNuD
c050994995
Fix PPTC on Windows 7. (#1369)
* Fix PPTC on Windows 7.

* Address gdkchan comment.
2020-07-09 10:45:24 +10:00
riperiperi
9a49f8aec9
Fix VMVN (immediate), Add VPMIN, VPMAX, VMVN (register) (#1303)
* Add Vmvn (register), tests for both Vmvn variants.

* Add Vpmin, Vpmax, improve Non-FastFp accuracy for Vpadd

* Rebase on top of PTC.

* Add Nopcode

* Increment PTC version.

* Fix nits.
2020-06-24 10:43:44 +10:00
Thog
05c63f3061
Increment PTC version (#1311)
Fix issues caused by 2421186d97
2020-06-18 13:41:00 +02:00
Ficture Seven
2421186d97
Generalize tail continues (#1298)
* Generalize tail continues

* Fix DecodeBasicBlock

`Next` and `Branch` would be null, which is not the state expected by
the branch instructions. They end up branching or falling into a block
which is never populated by the `Translator`. This causes an assert to
be fired when building the CFG.

* Clean up Decode overloads

* Do not synchronize when branching into exit block

If we're branching into an exit block, that exit block will tail
continue into another translation which already has a synchronization.

* Remove A32 predicate tail continue

If `block` is not an exit block then the `block.Next` must exist (as
per the last instruction of `block`).

* Throw if decoded 0 blocks

Address gdkchan's feedback

* Rebuild block list instead of setting to null

Address gdkchan's feedback
2020-06-18 13:37:21 +10:00
LDj3SNuD
5e724cf24e
Add Profiled Persistent Translation Cache. (#769)
* Delete DelegateTypes.cs

* Delete DelegateCache.cs

* Add files via upload

* Update Horizon.cs

* Update Program.cs

* Update MainWindow.cs

* Update Aot.cs

* Update RelocEntry.cs

* Update Translator.cs

* Update MemoryManager.cs

* Update InstEmitMemoryHelper.cs

* Update Delegates.cs

* Nit.

* Nit.

* Nit.

* 10 fewer MSIL bytes for us

* Add comment. Nits.

* Update Translator.cs

* Update Aot.cs

* Nits.

* Opt..

* Opt..

* Opt..

* Opt..

* Allow to change compression level.

* Update MemoryManager.cs

* Update Translator.cs

* Manage corner cases during the save phase. Nits.

* Update Aot.cs

* Translator response tweak for Aot disabled. Nit.

* Nit.

* Nits.

* Create DelegateHelpers.cs

* Update Delegates.cs

* Nit.

* Nit.

* Nits.

* Fix due to #784.

* Fixes due to #757 & #841.

* Fix due to #846.

* Fix due to #847.

* Use MethodInfo for managed method calls.

Use IR methods instead of managed methods about Max/Min (S/U).
Follow-ups & Nits.

* Add missing exception messages.

Reintroduce slow path for Fmov_Vi.
Implement slow path for Fmov_Si.

* Switch to the new folder structure.

Nits.

* Impl. index-based relocation information. Impl. cache file version field.

* Nit.

* Address gdkchan comments.

Mainly:
- fixed cache file corruption issue on exit; - exposed a way to disable AOT on the GUI.

* Address AcK77 comment.

* Address Thealexbarney, jduncanator & emmauss comments.

Header magic, CpuId (FI) & Aot -> Ptc.

* Adaptation to the new application reloading system.

Improvements to the call system of managed methods.
Follow-ups.
Nits.

* Get the same boot times as on master when PTC is disabled.

* Profiled Aot.

* A32 support (#897).

* #975 support (1 of 2).

* #975 support (2 of 2).

* Rebase fix & nits.

* Some fixes and nits (still one bug left).

* One fix & nits.

* Tests fix (by gdk) & nits.

* Support translations not only in high quality and rejit.

Nits.

* Added possibility to skip translations and continue execution, using `ESC` key.

* Update SettingsWindow.cs

* Update GLRenderer.cs

* Update Ptc.cs

* Disabled Profiled PTC by default as requested in the past by gdk.

* Fix rejit bug. Increased number of parallel translations. Add stack unwinding stuffs support (1 of 2).

Nits.

* Add stack unwinding stuffs support (2 of 2). Tuned number of parallel translations.

* Restored the ability to assemble jumps with 8-bit offset when Profiled PTC is disabled or during profiling.

Modifications due to rebase.
Nits.

* Limited profiling of the functions to be translated to the addresses belonging to the range of static objects only.

* Nits.

* Nits.

* Update Delegates.cs

* Nit.

* Update InstEmitSimdArithmetic.cs

* Address riperiperi comments.

* Fixed the issue of unjustifiably longer boot times at the second boot than at the first boot, measured at the same time or reference point and with the same number of translated functions.

* Implemented a simple redundant load/save mechanism.

Halved the value of Decoder.MaxInstsPerFunction more appropriate for the current performance of the Translator.
Replaced by Logger.PrintError to Logger.PrintDebug in TexturePool.cs about the supposed invalid texture format to avoid the spawn of the log.
Nits.

* Nit.

Improved Logger.PrintError in TexturePool.cs to avoid log spawn.
Added missing code for FZ handling (in output) for fp max/min instructions (slow paths).

* Add configuration migration for PTC

Co-authored-by: Thog <me@thog.eu>
2020-06-16 20:28:02 +02:00
LDj3SNuD
3b70a28087
Unwinding Follow-up. Fix a bug in JitUnwindWindows where ... (#1238)
... in case of "Vector" unwind codes the remaining unwind codes could be corrupted.
Nits.
2020-05-15 13:46:35 +02:00
gdkchan
96c7988671
Remove CpuId IR instruction (#1227) 2020-05-13 15:30:21 +10:00
Ficture Seven
4b79b78b8c
Fix tailcall case in EmitterContext (#1235) 2020-05-11 10:04:52 +10:00
Ficture Seven
180ad8605d
Improve IRDumper (#1135)
* Improve IRDumper

* Make Symbols.Add(ulong, ulong, ulong, string) thread safe

* Use a StringBuilder for MemoryOperand

* Add #if M_DEBUG guards

* Fix JMP_TABLE typo

* Fix using in Symbols

* Use Conditional("M_DEBUG") instead

Address gdkchan's feedback

* Use a struct instead of 4-tuple

Address gdkchan's feedback

* Place symbols in comments instead

Address gdkchan's feedback

* Use StringBuilder throughout

* Handle offsetted symbols

* Fix naming convention of Builder

* Avoid ArgumentException

* Remove unnecessary using

* Use switch expression instead

* Turn into a class

* Clean up

* Remove unnecessary using
2020-05-04 12:06:22 +10:00
gdkchan
f77694e4f7
Implement a new physical memory manager and replace DeviceMemory (#856)
* Implement a new physical memory manager and replace DeviceMemory

* Proper generic constraints

* Fix debug build

* Add memory tests

* New CPU memory manager and general code cleanup

* Remove host memory management from CPU project, use Ryujinx.Memory instead

* Fix tests

* Document exceptions on MemoryBlock

* Fix leak on unix memory allocation

* Proper disposal of some objects on tests

* Fix JitCache not being set as initialized

* GetRef without checks for 8-bits and 16-bits CAS

* Add MemoryBlock destructor

* Throw in separate method to improve codegen

* Address PR feedback

* QueryModified improvements

* Fix memory write tracking not marking all pages as modified in some cases

* Simplify MarkRegionAsModified

* Remove XML doc for ghost param

* Add back optimization to avoid useless buffer updates

* Add Ryujinx.Cpu project, move MemoryManager there and remove MemoryBlockWrapper

* Some nits

* Do not perform address translation when size is 0

* Address PR feedback and format NativeInterface class

* Remove ghost parameter description

* Update Ryujinx.Cpu to .NET Core 3.1

* Address PR feedback

* Fix build

* Return a well defined value for GetPhysicalAddress with invalid VA, and do not return unmapped ranges as modified

* Typo
2020-05-04 08:54:50 +10:00
Ficture Seven
71dbb38b9a
Do not compute dominance information when not in SSA (#1176) 2020-05-01 23:15:55 +02:00
Chenj168
561d64e5bf
Modify TranslatedFunction.GetPointer () to optimize performance (#995)
* add local var to reduce calling Marshal.GetFunctionPointerForDelegate

* modify code style
2020-03-20 09:11:20 +11:00
riperiperi
8226997bc7
CodeGen Optimisations (LSRA and Translator) (#978)
* Start of JIT garbage collection improvements

- thread static pool for Operand, MemoryOperand, Operation
- Operands and Operations are always to be constructed via their static
helper classes, so they can be pooled.
- removing LinkedList from Node for sources/destinations (replaced with
List<>s for now, but probably could do arrays since size is bounded)
- removing params constructors from Node
- LinkedList<> to List<> with Clear() for Operand assignments/uses
- ThreadStaticPool is very simple and basically just exists for the
purpose of our specific translation allocation problem. Right now it
will stay at the worst case allocation count for that thread (so far) -
the pool can never shrink.

- Still some cases of Operand[] that haven't been removed yet. Will need
to evaluate them (eg. is there a reasonable max number of params for
Calls?)

* ConcurrentStack instead of ConcurrentQueue for Rejit

* Optimize some parts of LSRA

- BitMap now operates on 64-bit int rather than 32-bit
- BitMap is now pooled in a ThreadStatic pool (within lrsa)
- BitMap now is now its own iterator. Marginally speeds up iterating
through the bits.
- A few cases where enumerators were generated have been converted to
forms that generate less garbage.
- New data structure for sorting _usePositions in LiveIntervals. Much
faster split, NextUseAfter, initial insertion. Random insertion is
slightly slower.
- That last one is WIP since you need to insert the values backwards. It
would be ideal if it just flipped it for you, uncomplicating things on
the caller side.

* Use a static pool of thread static pools. (yes.)

Prevents each execution thread creating its own lowCq pool and making me cry.

* Move constant value to top, change naming convention.

* Fix iteration of memory operands.

* Increase max thread count.

* Address Feedback
2020-03-18 22:44:32 +11:00
riperiperi
d904706fc0
Use a Jump Table for direct and indirect calls/jumps, removing transitions to managed (#975)
* Implement Jump Table for Native Calls

NOTE: this slows down rejit considerably! Not recommended to be used
without codegen optimisation or AOT.

- Does not work on Linux
- A32 needs an additional commit.

* A32 Support

(WIP)

* Actually write Direct Call pointers to the table

That would help.

* Direct Calls: Rather than returning to the translator, attempt to keep within the native stack frame.

A return to the translator can still happen, but only by exceptionally
bubbling up to it.

Also:
- Always translate lowCq as a function. Faster interop with the direct
jumps, and this will be useful in future if we want to do speculative
translation.
- Tail Call Detection: after the decoding stage, detect if we do a tail
call, and avoid translating into it. Detected if a jump is made to an
address outwith the contiguous sequence of blocks surrounding the entry
point. The goal is to reduce code touched by jit and rejit.

* A32 Support

* Use smaller max function size for lowCq, fix exceptional returns

When a return has an unexpected value and there is no code block
following this one, we now return the value rather than continuing.

* CompareAndSwap (buggy)

* Ensure CompareAndSwap does not get optimized away.

* Use CompareAndSwap to make the dynamic table thread safe.

* Tail call for linux, throw on too many arguments.

* Combine CompareAndSwap 128 and 32/64.

They emit different IR instructions since their PreAllocator behaviour
is different, but now they just have one function on EmitterContext.

* Fix issues separating from optimisations.

* Use a stub to find and execute missing functions.

This allows us to skip doing many runtime comparisons and branches, and reduces the amount of code we need to emit significantly.

For the indirect call table, this stub also does the work of moving in the highCq address to the table when one is found.

* Make Jump Tables and Jit Cache dynmically resize

Reserve virtual memory, commit as needed.

* Move TailCallRemover to its own class.

* Multithreaded Translation (based on heuristic)

A poor one, at that. Need to get core count for a better one, which
means a lot of OS specific garbage.

* Better priority management for background threads.

* Bound core limit a bit more

Past a certain point the load is not paralellizable and starts stealing from the main thread. Likely due to GC, memory, heap allocation thread contention. Reduce by one core til optimisations come to improve the situation.

* Fix memory management on linux.

* Temporary solution to some sync problems.

This will make sure threads exit correctly, most of the time. There is a potential race where setting the sync counter to 0 does nothing (counter stays at what it was before, thread could take too long to exit), but we need to find a better way to do this anyways. Synchronization frequency has been tightened as we never enter blockwise segments of code. Essentially this means, check every x functions or loop iterations, before lowcq blocks existed and were worth just as much. Ideally it should be done in a better way, since functions can be anywhere from 1 to 5000 instructions. (maybe based on host timer, or an interrupt flag from a scheduler thread)

* Address feedback minus CompareAndSwap change.

* Use default ReservedRegion granularity.

* Merge CompareAndSwap with its V128 variant.

* We already got the source, no need to do it again.

* Make sure all background translation threads exit.

* Fix CompareAndSwap128

Detection criteria was a bit scuffed.

* Address Comments.
2020-03-12 14:20:55 +11:00
gdkchan
61d79facd1
Optimize x64 loads and stores using complex addressing modes (#972)
* Optimize x64 loads and stores using complex addressing modes

* This was meant to be used for testing
2020-03-10 09:29:34 +11:00
riperiperi
b1b6f294f2
Add most of the A32 instruction set to ARMeilleure (#897)
* Implement TEQ and MOV (Imm16)

* Initial work on A32 instructions + SVC. No tests yet, hangs in rtld.

* Implement CLZ, fix BFI and BFC

Now stops on SIMD initialization.

* Exclusive access instructions, fix to mul, system instructions.

Now gets to a break after SignalProcessWideKey64.

* Better impl of UBFX, add UDIV and SDIV

Now boots way further - now stuck on VMOV instruction.

* Many more instructions, start on SIMD and testing framework.

* Fix build issues

* svc: Rework 32 bit codepath

Fixing once and for all argument ordering issues.

* Fix 32 bits stacktrace

* hle debug: Add 32 bits dynamic section parsing

* Fix highCq mode, add many tests, fix some instruction bugs

Still suffers from critical malloc failure 😩

* Fix incorrect opcode decoders and a few more instructions.

* Add a few instructions and fix others. re-disable highCq for now.

Disabled the svc memory clear since i'm not sure about it.

* Fix build

* Fix typo in ordered/exclusive stores.

* Implement some more instructions, fix others.

Uxtab16/Sxtab16 are untested.

* Begin impl of pairwise, some other instructions.

* Add a few more instructions, a quick hack to fix svcs for now.

* Add tests and fix issues with VTRN, VZIP, VUZP

* Add a few more instructions, fix Vmul_1 encoding.

* Fix way too many instruction bugs, add tests for some of the more important ones.

* Fix HighCq, enable FastFP paths for some floating point instructions

(not entirely sure why these were disabled, so important to note this
commit exists)

Branching has been removed in A32 shifts until I figure out if it's
worth it

* Cleanup Part 1

There should be no functional change between these next few commits.
Should is the key word. (except for removing break handler)

* Implement 32 bits syscalls

Co-authored-by: riperiperi <rhy3756547@hotmail.com>

Implement all 32 bits counterparts of the 64 bits syscalls we currently
have.

* Refactor part 2: Move index/subindex logic to Operand

May have inadvertently fixed one (1) bug

* Add FlushProcessDataCache32

* Address jd's comments

* Remove 16 bit encodings from OpCodeTable

Still need to catch some edge cases (operands that use the "F" flag) and
make Q encodings with non-even indexes undefined.

* Correct Fpscr handling for FP vector slow paths

WIP

* Add StandardFPSCRValue behaviour for all Arithmetic instructions

* Add StandardFPSCRValue behaviour to compare instructions.

* Force passing of fpcr to FPProcessException and FPUnpack.

Reduces potential for code error significantly

* OpCode cleanup

* Remove urgency from DMB comment in MRRC

DMB is currently a no-op via the instruction, so it should likely still
be a no-op here.

* Test Cleanup

* Fix FPDefaultNaN on Ryzen CPUs

* Improve some tests, fix some shift instructions, add slow path for Vadd

* Fix Typo

* More test cleanup

* Flip order of Fx and index, to indicate that the operand's is the "base"

* Remove Simd32 register type, use Int32 and Int64 for scalars like A64 does.

* Reintroduce alignment to DecoderHelper (removed by accident)

* One more realign as reading diffs is hard

* Use I32 registers in A32 (part 2)

Swap default integer register type based on current execution mode.

* FPSCR flags as Registers (part 1)

Still need to change NativeContext and ExecutionContext to allow
getting/setting with the flag values.

* Use I32 registers in A32 (part 1)

* FPSCR flags as registers (part 2)

Only CMP flags are on the registers right now. It could be useful to use
more of the space in non-fast-float when implementing A32 flags
accurately in the fast path.

* Address Feedback

* Correct FP->Int behaviour (should saturate)

* Make branches made by writing to PC eligible for Rejit

Greatly improves performance in most games.

* Remove unused branching for Vtbl

* RejitRequest as a class rather than a tuple

Makes a lot more sense than storing tuples on a dictionary.

* Add VMOVN, VSHR (imm), VSHRN (imm) and related tests

* Re-order InstEmitSystem32

Alphabetical sorting.

* Address Feedback

Feedback from Ac_K, remove and sort usings.

* Address Feedback 2

* Address Feedback from LDj3SNuD

Opcode table reordered to have alphabetical sorting within groups,
Vmaxnm and Vminnm have split names to be less ambiguous, SoftFloat nits,
Test nits and Test simplification with ValueSource.

* Add Debug Asserts to A32 helpers

Mainly to prevent the shift ones from being used on I64 operands, as
they expect I32 input for most operations (eg. carry flag setting), and
expect I32 input for shift and boolean amounts. Most other helper
functions don't take Operands, throw on out of range values, and take
specific types of OpCode, so didn't need any asserts.

* Use ConstF rather than creating an operand.

(useful for pooling in future)

* Move exclusive load to helper, reference call flag rather than literal 1.

* Address LDj feedback (minus table flatten)

one final look before it's all gone. the world is so beautiful.

* Flatten OpCodeTable

oh no

* Address more table ordering

* Call Flag as int on A32

Co-authored-by: Natalie C. <cyuubiapps@gmail.com>
Co-authored-by: Thog <thog@protonmail.com>
2020-02-24 08:20:40 +11:00
gdkchan
e5f78fb1d4
Replace LinkedList by IntrusiveList to avoid allocations on JIT (#931)
* Replace LinkedList by IntrusiveList to avoid allocations on JIT

* Fix wrong replacements
2020-02-17 22:30:54 +01:00
Ac_K
5facc0c07f Name all threads (#886)
* Name all threads

Close #874

* use ThreadName instead of ThreadId in Logging
2020-01-13 01:21:54 +01:00
LDj3SNuD
eee639d6ba .NET Core 3.0 is here! (#784)
* .NET Core 3.0 is here!

* Remove IMemoryManager.cs and its references.

* Add T Math/F.FusedMultiplyAdd(T, T, T). Nits.

* Nit.

* Update appveyor.yml

* Revert "Resolve Visual Studio build issues"

This reverts commit 1772128ce0.

* Update SvcTable.cs
2019-10-31 19:09:03 +01:00
gdkchan
a731ab3a2a Add a new JIT compiler for CPU code (#693)
* Start of the ARMeilleure project

* Refactoring around the old IRAdapter, now renamed to PreAllocator

* Optimize the LowestBitSet method

* Add CLZ support and fix CLS implementation

* Add missing Equals and GetHashCode overrides on some structs, misc small tweaks

* Implement the ByteSwap IR instruction, and some refactoring on the assembler

* Implement the DivideUI IR instruction and fix 64-bits IDIV

* Correct constant operand type on CSINC

* Move division instructions implementation to InstEmitDiv

* Fix destination type for the ConditionalSelect IR instruction

* Implement UMULH and SMULH, with new IR instructions

* Fix some issues with shift instructions

* Fix constant types for BFM instructions

* Fix up new tests using the new V128 struct

* Update tests

* Move DIV tests to a separate file

* Add support for calls, and some instructions that depends on them

* Start adding support for SIMD & FP types, along with some of the related ARM instructions

* Fix some typos and the divide instruction with FP operands

* Fix wrong method call on Clz_V

* Implement ARM FP & SIMD move instructions, Saddlv_V, and misc. fixes

* Implement SIMD logical instructions and more misc. fixes

* Fix PSRAD x86 instruction encoding, TRN, UABD and UABDL implementations

* Implement float conversion instruction, merge in LDj3SNuD fixes, and some other misc. fixes

* Implement SIMD shift instruction and fix Dup_V

* Add SCVTF and UCVTF (vector, fixed-point) variants to the opcode table

* Fix check with tolerance on tester

* Implement FP & SIMD comparison instructions, and some fixes

* Update FCVT (Scalar) encoding on the table to support the Half-float variants

* Support passing V128 structs, some cleanup on the register allocator, merge LDj3SNuD fixes

* Use old memory access methods, made a start on SIMD memory insts support, some fixes

* Fix float constant passed to functions, save and restore non-volatile XMM registers, other fixes

* Fix arguments count with struct return values, other fixes

* More instructions

* Misc. fixes and integrate LDj3SNuD fixes

* Update tests

* Add a faster linear scan allocator, unwinding support on windows, and other changes

* Update Ryujinx.HLE

* Update Ryujinx.Graphics

* Fix V128 return pointer passing, RCX is clobbered

* Update Ryujinx.Tests

* Update ITimeZoneService

* Stop using GetFunctionPointer as that can't be called from native code, misc. fixes and tweaks

* Use generic GetFunctionPointerForDelegate method and other tweaks

* Some refactoring on the code generator, assert on invalid operations and use a separate enum for intrinsics

* Remove some unused code on the assembler

* Fix REX.W prefix regression on float conversion instructions, add some sort of profiler

* Add hardware capability detection

* Fix regression on Sha1h and revert Fcm** changes

* Add SSE2-only paths on vector extract and insert, some refactoring on the pre-allocator

* Fix silly mistake introduced on last commit on CpuId

* Generate inline stack probes when the stack allocation is too large

* Initial support for the System-V ABI

* Support multiple destination operands

* Fix SSE2 VectorInsert8 path, and other fixes

* Change placement of XMM callee save and restore code to match other compilers

* Rename Dest to Destination and Inst to Instruction

* Fix a regression related to calls and the V128 type

* Add an extra space on comments to match code style

* Some refactoring

* Fix vector insert FP32 SSE2 path

* Port over the ARM32 instructions

* Avoid memory protection races on JIT Cache

* Another fix on VectorInsert FP32 (thanks to LDj3SNuD

* Float operands don't need to use the same register when VEX is supported

* Add a new register allocator, higher quality code for hot code (tier up), and other tweaks

* Some nits, small improvements on the pre allocator

* CpuThreadState is gone

* Allow changing CPU emulators with a config entry

* Add runtime identifiers on the ARMeilleure project

* Allow switching between CPUs through a config entry (pt. 2)

* Change win10-x64 to win-x64 on projects

* Update the Ryujinx project to use ARMeilleure

* Ensure that the selected register is valid on the hybrid allocator

* Allow exiting on returns to 0 (should fix test regression)

* Remove register assignments for most used variables on the hybrid allocator

* Do not use fixed registers as spill temp

* Add missing namespace and remove unneeded using

* Address PR feedback

* Fix types, etc

* Enable AssumeStrictAbiCompliance by default

* Ensure that Spill and Fill don't load or store any more than necessary
2019-08-08 21:56:22 +03:00