Whenever the API accepts an AdditionalBytes, it could potentially
overflow the size if additional bytes is huge. Limiting it to uint32_t
should suffice to prevent overflows on 64-bit machines. On 32-bit
machines we explicitly check for overflows.
Bug: 572627513
Change-Id: I6ccd57a6a1b9d02ba4644be2113602081fdbd9c0
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/8567760
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/main@{#110706}
To improve allocation performance, this CL avoids setting the object
start bit in the ObjectStartBitmap during fast-path LAB allocations for
regular (non-mixin) objects.
Since the concurrent/incremental marker only needs to perform
inner-pointer resolution for mixin types, we can safely omit setting
the start bit for non-mixin objects during allocation. The bitmap is
instead repaired on-the-fly by the mutator when needed, and fully
populated during sweeping.
Detailed changes:
1. Allocation Optimization: Removed the `SetBit` call in `ObjectAllocator`
for LAB allocations, except when `may_contain_mixins == kYes`. Mixins
continue to set their start bit immediately to ensure they can be
reliably found by the concurrent marker.
2. On-the-fly Repair (Mutator): `BasePage::ObjectHeaderFromInnerAddress`
now detects if the bitmap has gaps and repairs it on-the-fly by
linearizing the bitmap from the last known set bit up to the requested
address. This is safe as it is only called by the mutator.
3. Concurrent Lookup (Marker): Introduced
`BasePage::ObjectHeaderFromInnerAddressConcurrent` for the concurrent
marker. To avoid data races with the mutator, it strictly reads from
the bitmap and does not attempt to repair it. This is safe because the
concurrent marker only resolves inner pointers for mixin objects, which
are guaranteed to have had their start bits set upon allocation.
4. Sweeper Updates: The sweeper now populates the start bit for all
live objects and free-list entries, restoring the bitmap to a fully
consistent state at the end of the sweep phase.
Design Doc: https://docs.google.com/document/d/1i4Sx-p6a_toYLwEt4HnQeJ9Xm28FmevwT2e5DzkGIaQ/edit?usp=sharing
Bug: 516682115
Change-Id: Ib8a3dfd28778d3b07beb5bf8f38331861b384b6b
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/7828550
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/main@{#107560}
The GCInfoIndex is always written on allocation. This does not affect
writing/reading the construction bit which happens with a mask.
This safes one shift in the allocation fast path (for GCInfoIndex).
Drive-by: Use std::atomic_ref instead of casts to std::atomic.
Change-Id: I8932f46ef582558d30eded9bab9b8d210a0ee1cb
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/7213182
Auto-Submit: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/main@{#104077}
This is a reland of commit 2c0facec78
The original CL was reverted due to a large binary size increase.
This CL doesn't fix the inlining, but only unexports unneeded
allocation functions.
Original change's description:
> cppgc: Force inline AllocationDispatcher::Invoke
>
> On ARM64 the compiler inserts "bti c" for these functions, which is
> unnecessary, since they get almost always inlined. The functions are
> currently exported (because they are nested in
> MakeGarbageCollectedTraitInternal, which is exported), so the compiler
> assumes they may be referenced from outside (through dlsym) and called
> indirectly, so it inserts "bti c" into them.
>
> Drive-by: Make MakeGarbageCollectedTraitInternal hidden
>
> Bug: 339075143
> Change-Id: I060a95aeac97c63a19fc52b14c2e03a7b98e6b2d
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/5521033
> Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
> Commit-Queue: Anton Bikineev <bikineev@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#93774}
Bug: 339075143
Change-Id: Ic3aab5d5ed3ab46009ce231f81106844d4bad97c
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/5522220
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Auto-Submit: Anton Bikineev <bikineev@chromium.org>
Cr-Commit-Position: refs/heads/main@{#93788}
This reverts commit 2c0facec78.
Reason for revert: breaks blink binary size increase: https://chromium-review.googlesource.com/c/chromium/src/+/5523279?checksPatchset=2&tab=checks)
Pleaase provide rationale
Original change's description:
> cppgc: Force inline AllocationDispatcher::Invoke
>
> On ARM64 the compiler inserts "bti c" for these functions, which is
> unnecessary, since they get almost always inlined. The functions are
> currently exported (because they are nested in
> MakeGarbageCollectedTraitInternal, which is exported), so the compiler
> assumes they may be referenced from outside (through dlsym) and called
> indirectly, so it inserts "bti c" into them.
>
> Drive-by: Make MakeGarbageCollectedTraitInternal hidden
>
> Bug: 339075143
> Change-Id: I060a95aeac97c63a19fc52b14c2e03a7b98e6b2d
> Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/5521033
> Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
> Commit-Queue: Anton Bikineev <bikineev@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#93774}
Bug: 339075143
Change-Id: I254f522ee45e3b64293932c2c36011a2d364706d
No-Presubmit: true
No-Tree-Checks: true
No-Try: true
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/5521815
Owners-Override: Francis McCabe <fgm@chromium.org>
Commit-Queue: Francis McCabe <fgm@chromium.org>
Auto-Submit: Francis McCabe <fgm@chromium.org>
Bot-Commit: Rubber Stamper <rubber-stamper@appspot.gserviceaccount.com>
Cr-Commit-Position: refs/heads/main@{#93777}
On ARM64 the compiler inserts "bti c" for these functions, which is
unnecessary, since they get almost always inlined. The functions are
currently exported (because they are nested in
MakeGarbageCollectedTraitInternal, which is exported), so the compiler
assumes they may be referenced from outside (through dlsym) and called
indirectly, so it inserts "bti c" into them.
Drive-by: Make MakeGarbageCollectedTraitInternal hidden
Bug: 339075143
Change-Id: I060a95aeac97c63a19fc52b14c2e03a7b98e6b2d
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/5521033
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Cr-Commit-Position: refs/heads/main@{#93774}
Adds support for double-word aligned, i.e., 8 bytes on 32-bit
platforms and 16 bytes on 64-bit platforms, objects in Oilpan.
Changes:
- Adds generic alignment APIs and overrides.
- Internal logic to support double-word aligned allocations on LABs.
- Adjusts natural alignment of large objects to follow double-word.
- Adds a new static_assert() that suggests users file a bug if higher
alignment is required.
- Statically checks that no allocations with non-default alignment
target custom spaces that support compaction.
Bug: v8:12295
Change-Id: I05766ce2349055d5d78b68919be00e7ee91d5505
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3218150
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/main@{#77348}
In Blink's version of Oilpan, GCInfo objects would reside in .bss and
a table would translate between an index and the .bss address. Upon
retrieving a GCInfoIndex, the slow path merely passes a .bss pointer
to a slow path setup method to create the table mapping.
In cppgc, we set up GCInfo entries directly in the table. This is
slightly faster for actually using GCInfo objects as there's no
indirection between table and .bss, and it also saves one pointer (the
indirection) per type that is set up. The downside of this approach is
that individual components of a GCInfo objects, that are all
type-dependent, need to be passed to the conditional setup method.
Since GCInfo indices must be retrieved on each allocation, this
pollutes the fast path with additional instructions.
However, GCInfo components are actually known at compile-time for many
objects. In such cases, we can use a compile-time static dispatch to
encode the known parameters in different functions. This saves around
40KiB of memory on ChromePublic.apk and also creates a more compact
fast path for allocation.
Bug: chromium:1238884, chromium:1056170
Change-Id: Iedd809a8baefcc02f131d2b2c77d341b0abe43bb
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3094007
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#76291}
fetch_or (lock-prefixed-or on x86) is emulated with branching on
armv7/armv8 and therefore generates more instructions.
This improves android-binary-size by 45K. It should also improve
performance.
Bug: chromium:1238884
Change-Id: I48f59b645a4bb872b3798a1fde11608fd2930ce6
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/3090342
Commit-Queue: Anton Bikineev <bikineev@chromium.org>
Auto-Submit: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#76260}
The only valid way to define a GCed type T is by inheriting from
GarbageCollected<T>. Since this is prone to typos (see tests), add a
simple check that covers most interesting use cases.
The static assert covers
A -> B -> GarbageCollected<C>
The static assert does not cover
A -> B -> C -> GarbageCollected<B>
(In order to do so, we would need __direct_bases() support which is
not yet available for C++.)
Bug: pdfium:1670, chromium:1056170
Change-Id: I494de48992f8ba9a1f0f9daad60584d828717403
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2810415
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#73854}
Refactor SpacePolicy on a non-templated class to avoid the situation
of having MakeGarbageCollectedTraitBase<T>::SpacePolicy<U> refer to
different T and U which make it hard for the compiler to alias
anything.
Bug: chromium:1056170
Change-Id: I78eb0362d43403ad2712bcb65746eeb9f6ad44fa
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2769338
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#73494}
Adds GCInfo folding that delegates GCInfo requests to the
parent-most object if finalizer semantics match.
Folding is disabled for builds that want exact object names
as those names are also managed through GCInfo objects.
Bug: chromium:1056170
Change-Id: I783aad930587853741da533d0b9b56ba160d0596
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2748588
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#73339}
Unified heap support in V8 requires having another (at least internal)
heap that implements a unfied garbage collection strategy. This will
not re-use the already existing cppgc::Heap because there should be no
way in creating such a heap externally or scheduling stand-alone
garbage collections.
In order to have a common token, this CL introduces AllocationHandle
which can be passed to MakeGarbageCollected to allocate C++ objects.
V8 (soon) and the stand-alone heap both have methods to retrieve such
a handle.
This works around a problem with creating diamond class hierarchies
when a base class would be exposed on the public API level.
Fast paths for Blink are still possible because allocation handles can
be cached the same way (e.g. global, or TLS) as a heap can be cached.
Tbr: yangguo@chromium.org
Bug: chromium:1056170
Change-Id: I8e9472a2c24ef82d1178953e8429b1fd8a2344bc
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2238027
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Cr-Commit-Position: refs/heads/master@{#68310}
This patch provides infrastructure to pin object types to specific
spaces. This allows embedders to create mutual exclusive arenas for
certain (base) types. In future, this will also be used to provide
sliding-window compaction on certain custom spaces.
We mainly preserve the existing infrastructure with the difference
that spaces are now slightly more dynamic than in Blink as they are
kept in a vector instead of a fixed-size array.
The mechanism differs from Blink in that it does not allow the user
object to call allocation methods directly but instead provides a
trait that can be overridden to specify a custom space.
The patch preserves templatization for objects that do not go into
custom spaces to safe a branch in the allocation hot path.
Change-Id: I08aa6932348e2d6258e19c4a32d189865f459f02
Bug: chromium:1056170
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2187611
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67666}
This adds PostConstructionCallbackTrait which can be used to get a
callback that is executed right after an object instance is created.
This can be useful for hooks that require to be able to call into
virtual methods.
Bug: chromium:1074061
Change-Id: Idd5ef677fed291bcba81b9a47f2932c9bb5832b4
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2179385
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Anton Bikineev <bikineev@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67557}
- Fixes includes to be relative to include/ which allows embedders
to just add V8's include directory to get started.
- Adds public target for the library as "cppgc".
Bug: chromium:1056170
Change-Id: Iec9b644e20016a5d7281275b739821a050fd2540
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2157366
Reviewed-by: Michael Achenbach <machenbach@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#67278}
This CL adds basic infrastructure for:
- MakeGarbageCollected
- GarbageCollected and related type traits
- Heap (API / internal)
- Basic allocation based on malloc
- CollectGarbage without marking
This allows for allocation and reclamation through an explicit GC
call. No objects are held alive from any source (stack, globals,
refs), yet.
The exact wiring of platform is future work.
Change-Id: I81b7c0ba7b525188f8c0bf9de3b7af35d34322af
Bug: chromium:1056170
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2120538
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66887}
This adds HeapObjectHeader, a meta object that is put in front of
every managed object. HeapObjectHeader provides accessors for:
1. GCInfoIndex
2. In construction bit
3. size
4. Mark bit
Meta info is distributed among two uint16_t fields as (1.,2.) and
(3.,4.). This is convenient as the non-bit accessors (size,
GCInfoIndex) are constant during marking.
Object layout see heap-object-header.h.
Note: The current implementation does not bypass ASAN poisoning and
assumes an unpoisoned header whenever performing an access.
Bug: chromium:1056170
Change-Id: I753f15467ed5c2b22b47e64d3aa5a3c1baddf8e1
Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/2116031
Commit-Queue: Michael Lippautz <mlippautz@chromium.org>
Reviewed-by: Ulan Degenbaev <ulan@chromium.org>
Reviewed-by: Omer Katz <omerkatz@chromium.org>
Reviewed-by: Hannes Payer <hpayer@chromium.org>
Cr-Commit-Position: refs/heads/master@{#66861}
Joyee Cheung
Omer Katz
Anton Bikineev
Michael Lippautz
Jakob Kummerow
Francis McCabe