Quoting: slaapliedje

Quoting: F.Ultra

Quoting: slaapliedje

Quoting: F.Ultra

Quoting: Guest

Btrfs file system performance improvements.

Is long mounting of large HDD partitions fixed now?

What counts as large partitions and long time? I use BTRFS on several servers each with 153TB per partition and mounting is sub second and have been for many years.

edit: that said one of the listed items is improved mount times on large systems:

Hi,

please pull the following updates for btrfs. There's a bunch of
performance improvements, most notably the FIEMAP speedup, the new block
group tree to speed up mount on large filesystems, more io_uring
integration, some sysfs exports and the usual fixes and core updates.

Thanks.

---

Performance:

- outstanding FIEMAP speed improvement
- algorithmic change how extents are enumerated leads to orders of
magnitude speed boost (uncached and cached)
- extent sharing check speedup (2.2x uncached, 3x cached)
- add more cancellation points, allowing to interrupt seeking in files
with large number of extents
- more efficient hole and data seeking (4x uncached, 1.3x cached)
- sample results:
256M, 32K extents: 4s -> 29ms (~150x)
512M, 64K extents: 30s -> 59ms (~550x)
1G, 128K extents: 225s -> 120ms (~1800x)

- improved inode logging, especially for directories (on dbench workload
throughput +25%, max latency -21%)

- improved buffered IO, remove redundant extent state tracking, lowering
memory consumption and avoiding rb tree traversal

- add sysfs tunable to let qgroup temporarily skip exact accounting when
deleting snapshot, leading to a speedup but requiring a rescan after
that, will be used by snapper

- support io_uring and buffered writes, until now it was just for direct
IO, with the no-wait semantics implemented in the buffered write path
it now works and leads to speed improvement in IOPS (2x), throughput
(2.2x), latency (depends, 2x to 150x)

- small performance improvements when dropping and searching for extent
maps as well as when flushing delalloc in COW mode (throughput +5MB/s)

User visible changes:

- new incompatible feature block-group-tree adding a dedicated tree for
tracking block groups, this allows a much faster load during mount and
avoids seeking unlike when it's scattered in the extent tree items
- this reduces mount time for many-terabyte sized filesystems
- conversion tool will be provided so existing filesystem can also be
updated in place
- to reduce test matrix and feature combinations requires no-holes
and free-space-tree (mkfs defaults since 5.15)

- improved reporting of super block corruption detected by scrub

- scrub also tries to repair super block and does not wait until next
commit

- discard stats and tunables are exported in sysfs
(/sys/fs/btrfs/FSID/discard)

- qgroup status is exported in sysfs (/sys/sys/fs/btrfs/FSID/qgroups/)

- verify that super block was not modified when thawing filesystem

Fixes:

- FIEMAP fixes
- fix extent sharing status, does not depend on the cached status where
merged
- flush delalloc so compressed extents are reported correctly

- fix alignment of VMA for memory mapped files on THP

- send: fix failures when processing inodes with no links (orphan files
and directories)

- fix race between quota enable and quota rescan ioctl

- handle more corner cases for read-only compat feature verification

- fix missed extent on fsync after dropping extent maps

Core:

- lockdep annotations to validate various transactions states and state
transitions

- preliminary support for fs-verity in send

- more effective memory use in scrub for subpage where sector is smaller
than page

- block group caching progress logic has been removed, load is now
synchronous

- simplify end IO callbacks and bio handling, use chained bios instead
of own tracking

- add no-wait semantics to several functions (tree search, nocow,
flushing, buffered write

- cleanups and refactoring

MM changes:

- export balance_dirty_pages_ratelimited_flags

I wonder how long it would take me to fill up 153TB with my Steam Library on my 2gbit fiber line...

Well if you start at 0 and then manages to fully saturate that 2Gbps line of yours and pay for all the games, and being able to do so in a manner that would keep the line saturated still and if we assume that this line would be dedicated to just the downloading of said games and not for other Internet usages (such as e.g visiting the Steam store) then it would take aprox 8 days.

Haha! I mean I likely have enough in my steam library where I could fill up that much. Too many games that are 100GB+ these days... hell, the libraries of most of the 16/32bit era aren't even close to 100GB.

Imagine how worse it will become once they decide to ship textures for 8K gaming...

View this comment - View article - View full comments

Quoting: slaapliedje

Quoting: F.Ultra

Quoting: Guest

Btrfs file system performance improvements.

Is long mounting of large HDD partitions fixed now?

What counts as large partitions and long time? I use BTRFS on several servers each with 153TB per partition and mounting is sub second and have been for many years.

edit: that said one of the listed items is improved mount times on large systems:

Hi,

please pull the following updates for btrfs. There's a bunch of
performance improvements, most notably the FIEMAP speedup, the new block
group tree to speed up mount on large filesystems, more io_uring
integration, some sysfs exports and the usual fixes and core updates.

Thanks.

---

Performance:

- outstanding FIEMAP speed improvement
- algorithmic change how extents are enumerated leads to orders of
magnitude speed boost (uncached and cached)
- extent sharing check speedup (2.2x uncached, 3x cached)
- add more cancellation points, allowing to interrupt seeking in files
with large number of extents
- more efficient hole and data seeking (4x uncached, 1.3x cached)
- sample results:
256M, 32K extents: 4s -> 29ms (~150x)
512M, 64K extents: 30s -> 59ms (~550x)
1G, 128K extents: 225s -> 120ms (~1800x)

- improved inode logging, especially for directories (on dbench workload
throughput +25%, max latency -21%)

- improved buffered IO, remove redundant extent state tracking, lowering
memory consumption and avoiding rb tree traversal

- add sysfs tunable to let qgroup temporarily skip exact accounting when
deleting snapshot, leading to a speedup but requiring a rescan after
that, will be used by snapper

- support io_uring and buffered writes, until now it was just for direct
IO, with the no-wait semantics implemented in the buffered write path
it now works and leads to speed improvement in IOPS (2x), throughput
(2.2x), latency (depends, 2x to 150x)

- small performance improvements when dropping and searching for extent
maps as well as when flushing delalloc in COW mode (throughput +5MB/s)

User visible changes:

- new incompatible feature block-group-tree adding a dedicated tree for
tracking block groups, this allows a much faster load during mount and
avoids seeking unlike when it's scattered in the extent tree items
- this reduces mount time for many-terabyte sized filesystems
- conversion tool will be provided so existing filesystem can also be
updated in place
- to reduce test matrix and feature combinations requires no-holes
and free-space-tree (mkfs defaults since 5.15)

- improved reporting of super block corruption detected by scrub

- scrub also tries to repair super block and does not wait until next
commit

- discard stats and tunables are exported in sysfs
(/sys/fs/btrfs/FSID/discard)

- qgroup status is exported in sysfs (/sys/sys/fs/btrfs/FSID/qgroups/)

- verify that super block was not modified when thawing filesystem

Fixes:

- FIEMAP fixes
- fix extent sharing status, does not depend on the cached status where
merged
- flush delalloc so compressed extents are reported correctly

- fix alignment of VMA for memory mapped files on THP

- send: fix failures when processing inodes with no links (orphan files
and directories)

- fix race between quota enable and quota rescan ioctl

- handle more corner cases for read-only compat feature verification

- fix missed extent on fsync after dropping extent maps

Core:

- lockdep annotations to validate various transactions states and state
transitions

- preliminary support for fs-verity in send

- more effective memory use in scrub for subpage where sector is smaller
than page

- block group caching progress logic has been removed, load is now
synchronous

- simplify end IO callbacks and bio handling, use chained bios instead
of own tracking

- add no-wait semantics to several functions (tree search, nocow,
flushing, buffered write

- cleanups and refactoring

MM changes:

- export balance_dirty_pages_ratelimited_flags

I wonder how long it would take me to fill up 153TB with my Steam Library on my 2gbit fiber line...

Well if you start at 0 and then manages to fully saturate that 2Gbps line of yours and pay for all the games, and being able to do so in a manner that would keep the line saturated still and if we assume that this line would be dedicated to just the downloading of said games and not for other Internet usages (such as e.g visiting the Steam store) then it would take aprox 8 days.

View this comment - View article - View full comments

Quoting: Guest

Quoting: F.Ultra

Quoting: Guest

Btrfs file system performance improvements.

Is long mounting of large HDD partitions fixed now?

What counts as large partitions and long time? I use BTRFS on several servers each with 153TB per partition and mounting is sub second and have been for many years.

edit: that said one of the listed items is improved mount times on large systems:

Hi,

please pull the following updates for btrfs. There's a bunch of
performance improvements, most notably the FIEMAP speedup, the new block
group tree to speed up mount on large filesystems, more io_uring
integration, some sysfs exports and the usual fixes and core updates.

Thanks.

---

Performance:

- outstanding FIEMAP speed improvement
- algorithmic change how extents are enumerated leads to orders of
magnitude speed boost (uncached and cached)
- extent sharing check speedup (2.2x uncached, 3x cached)
- add more cancellation points, allowing to interrupt seeking in files
with large number of extents
- more efficient hole and data seeking (4x uncached, 1.3x cached)
- sample results:
256M, 32K extents: 4s -> 29ms (~150x)
512M, 64K extents: 30s -> 59ms (~550x)
1G, 128K extents: 225s -> 120ms (~1800x)

- improved inode logging, especially for directories (on dbench workload
throughput +25%, max latency -21%)

- improved buffered IO, remove redundant extent state tracking, lowering
memory consumption and avoiding rb tree traversal

- add sysfs tunable to let qgroup temporarily skip exact accounting when
deleting snapshot, leading to a speedup but requiring a rescan after
that, will be used by snapper

- support io_uring and buffered writes, until now it was just for direct
IO, with the no-wait semantics implemented in the buffered write path
it now works and leads to speed improvement in IOPS (2x), throughput
(2.2x), latency (depends, 2x to 150x)

- small performance improvements when dropping and searching for extent
maps as well as when flushing delalloc in COW mode (throughput +5MB/s)

User visible changes:

- new incompatible feature block-group-tree adding a dedicated tree for
tracking block groups, this allows a much faster load during mount and
avoids seeking unlike when it's scattered in the extent tree items
- this reduces mount time for many-terabyte sized filesystems
- conversion tool will be provided so existing filesystem can also be
updated in place
- to reduce test matrix and feature combinations requires no-holes
and free-space-tree (mkfs defaults since 5.15)

- improved reporting of super block corruption detected by scrub

- scrub also tries to repair super block and does not wait until next
commit

- discard stats and tunables are exported in sysfs
(/sys/fs/btrfs/FSID/discard)

- qgroup status is exported in sysfs (/sys/sys/fs/btrfs/FSID/qgroups/)

- verify that super block was not modified when thawing filesystem

Fixes:

- FIEMAP fixes
- fix extent sharing status, does not depend on the cached status where
merged
- flush delalloc so compressed extents are reported correctly

- fix alignment of VMA for memory mapped files on THP

- send: fix failures when processing inodes with no links (orphan files
and directories)

- fix race between quota enable and quota rescan ioctl

- handle more corner cases for read-only compat feature verification

- fix missed extent on fsync after dropping extent maps

Core:

- lockdep annotations to validate various transactions states and state
transitions

- preliminary support for fs-verity in send

- more effective memory use in scrub for subpage where sector is smaller
than page

- block group caching progress logic has been removed, load is now
synchronous

- simplify end IO callbacks and bio handling, use chained bios instead
of own tracking

- add no-wait semantics to several functions (tree search, nocow,
flushing, buffered write

- cleanups and refactoring

MM changes:

- export balance_dirty_pages_ratelimited_flags

known issue for HDD (rotational disks)
https://bbs.archlinux.org/viewtopic.php?id=272376 [External Link]
https://lore.kernel.org/linux-btrfs/CAMQzBqCSzr4UO1VFTjtSDPt+0ukhf6yqK=q+eLA+Tp1hiB_weA@mail.gmail.com/t/#u [External Link]
https://lore.kernel.org/linux-btrfs/CAHQ7scVGPAwEGQOq3Kmn75GJzyzSQ9qrBBZrHFu+4YWQhGE0Lw@mail.gmail.com/t/#u [External Link]
That guy has 16TB disk and it takes under 1 minute to mount it.
I have 3.2TB partition and it mounts in 17 seconds, with defragmented extents.

Well I use rotational HDD:s as well (so ok they are SAS and not SATA drives but that shouldn't matter), in the latest machine I configured it was 36 x Seagate Exos X18 3.5 HDD 16TB SAS and it mounts almost instantly.

edit: not saying that it doesn't happen to people because it obviously are, just that it does not seem to be universal. Anyway here is to hoping that the changes in 6.1 solves it for the people that are experiencing this issue.

View this comment - View article - View full comments

Quoting: pleasereadthemanualEdit: And if you're familiar with the fable of Microsoft v Netscape, there's a monopoly that ended well for customers everywhere. Thanks to Microsoft, they standardized the idea of browsers everywhere being free-of-charge, which forced Netscape to pivot into Mozilla, which resulted in a great free software browser (that was also free-of-charge) that subsequently took market share from Internet Explorer because it was a better browser that supported open standards (some of which it had invented itself, like JavaScript) better than the competition.

If Netscape won, what would the world look like? Microsoft would have had to charge for Internet Explorer instead of bundling it with Windows 95 and future editions. Netscape wouldn't have been forced to innovate. Mozilla Firefox wouldn't exist, nor would any of Mozilla's other great free software programs.

The IE monopoly had zero to do with if a browser should be free or not, browsers had been free to download and use for some years before Microsoft decided to bundle IE with Windows and thus using their monopoly on the OS to gain leverage in the browser market. Nor was "give anyone a free browser" the aim of Microsoft, their goal was to shift enterprises to use their web server ISS instead of the Netscape Web Server (and in the end they both lost out to Apache anyway).

The problem was never that MS developed IE or that they released it for free, it was all about them abusing their monopoly. Fully agree that we probably in the end got it better with Firefox than if Netscape Navigator had survived (although we will never know since we don't have access to that parallel universe) but that is a "despite of" and not "thanks to" the market abuse by MS.

View this comment - View article - View full comments

Quoting: Guest

Btrfs file system performance improvements.

Is long mounting of large HDD partitions fixed now?

What counts as large partitions and long time? I use BTRFS on several servers each with 153TB per partition and mounting is sub second and have been for many years.

edit: that said one of the listed items is improved mount times on large systems:

Hi,

please pull the following updates for btrfs. There's a bunch of
performance improvements, most notably the FIEMAP speedup, the new block
group tree to speed up mount on large filesystems, more io_uring
integration, some sysfs exports and the usual fixes and core updates.

Thanks.

---

Performance:

- outstanding FIEMAP speed improvement
- algorithmic change how extents are enumerated leads to orders of
magnitude speed boost (uncached and cached)
- extent sharing check speedup (2.2x uncached, 3x cached)
- add more cancellation points, allowing to interrupt seeking in files
with large number of extents
- more efficient hole and data seeking (4x uncached, 1.3x cached)
- sample results:
256M, 32K extents: 4s -> 29ms (~150x)
512M, 64K extents: 30s -> 59ms (~550x)
1G, 128K extents: 225s -> 120ms (~1800x)

- improved inode logging, especially for directories (on dbench workload
throughput +25%, max latency -21%)

- improved buffered IO, remove redundant extent state tracking, lowering
memory consumption and avoiding rb tree traversal

- add sysfs tunable to let qgroup temporarily skip exact accounting when
deleting snapshot, leading to a speedup but requiring a rescan after
that, will be used by snapper

- support io_uring and buffered writes, until now it was just for direct
IO, with the no-wait semantics implemented in the buffered write path
it now works and leads to speed improvement in IOPS (2x), throughput
(2.2x), latency (depends, 2x to 150x)

- small performance improvements when dropping and searching for extent
maps as well as when flushing delalloc in COW mode (throughput +5MB/s)

User visible changes:

- new incompatible feature block-group-tree adding a dedicated tree for
tracking block groups, this allows a much faster load during mount and
avoids seeking unlike when it's scattered in the extent tree items
- this reduces mount time for many-terabyte sized filesystems
- conversion tool will be provided so existing filesystem can also be
updated in place
- to reduce test matrix and feature combinations requires no-holes
and free-space-tree (mkfs defaults since 5.15)

- improved reporting of super block corruption detected by scrub

- scrub also tries to repair super block and does not wait until next
commit

- discard stats and tunables are exported in sysfs
(/sys/fs/btrfs/FSID/discard)

- qgroup status is exported in sysfs (/sys/sys/fs/btrfs/FSID/qgroups/)

- verify that super block was not modified when thawing filesystem

Fixes:

- FIEMAP fixes
- fix extent sharing status, does not depend on the cached status where
merged
- flush delalloc so compressed extents are reported correctly

- fix alignment of VMA for memory mapped files on THP

- send: fix failures when processing inodes with no links (orphan files
and directories)

- fix race between quota enable and quota rescan ioctl

- handle more corner cases for read-only compat feature verification

- fix missed extent on fsync after dropping extent maps

Core:

- lockdep annotations to validate various transactions states and state
transitions

- preliminary support for fs-verity in send

- more effective memory use in scrub for subpage where sector is smaller
than page

- block group caching progress logic has been removed, load is now
synchronous

- simplify end IO callbacks and bio handling, use chained bios instead
of own tracking

- add no-wait semantics to several functions (tree search, nocow,
flushing, buffered write

- cleanups and refactoring

MM changes:

- export balance_dirty_pages_ratelimited_flags

View this comment - View article - View full comments

Quoting: massatt212

Quoting: F.Ultra

Quoting: Purple Library Guy

Quoting: massatt212i dont like it, they should allow Microsoft to acquire Activision, Sony is steady buying up single studios and everyone is worshipping it

I must have missed the existence of Sony churches. Really, I've never heard anyone say much good about Sony . . . does that actually happen?

Quoting: massatt212i dont like it, they should allow Microsoft to acquire Activision, Sony is steady buying up single studios and everyone is worshipping it, now Microsoft wants to compete its a problem, its so weird, i believe Blizzard will be in better hands with Microsoft compare to Activision.

Everyone saying microsoft if a Terrible company, just remember what Micheal Jackson said about sony, they both are bad.

This is not about which company is good or bad, this is about a merger having the potential to create a monopoly or a near monopoly in the market. So it also have zero to do with the amount of studios being bought (still MS have bought up more studios then Sony), instead it have to do with the huge size of Activision Blizzard, that is the single thing that activated FTC and why they are trying to block it. It's the size, nothing more.

No its not the size of Activision blizzard they are fighting about, its about the Call of duty Franchise, they dont care about the Blizzard games, its all about COD, cause if microsoft leaves says they dont want COD Sony wont have a problem with the acquirement.

You are talking about why SONY is concerned about the merger, I was talking about the FTC. The size of the deal is what made the governmental regulators like The FTC to start their investigation and they did this long before SONY said anything.

SOMY might be completely fine with the deal if COD is left out of the deal, The FTC will not. Key to this all is point #96 in their suit:

96. The Proposed Acquisition is reasonably likely to substantially lessen competition or tend to create a monopoly in the Relevant Markets by creating a combined firm with the ability and increased incentive to withhold Activision’s valuable gaming content from, or degrade Activision’s content for, Microsoft’s rivals. The combined firm would have the ability to exclude Microsoft’s rivals from access to some or all of Activision’s content in the Relevant Markets. Microsoft would have the power to decide if, when, and to what extent Activision content will be available on competing products. The Proposed Acquisition is likely to increase entry barriers, thereby dampening beneficial rivalry and innovation. If permitted to make Activision a captive supplier, Microsoft would have a substantially increased incentive to engage in strategies to that would likely lead to reduced consumer choice, higher prices or lower quality products, and less innovation.

Especially damning for MS is point #12 however as I understand it the EU commission have come out and clarified that they think that the FTC have gotten this wrong in that the EC allowed the deal without such a promise:

12. Microsoft’s past conduct provides a preview of the combined firm’s likely plans if it consummates the Proposed Acquisition, despite any assurances the company may offer regarding its plans. In March 2021, Microsoft acquired ZeniMax Media Inc. (“ZeniMax”), the parent company of the well-known game developer and publisher Bethesda Softworks LLC (“Bethesda”). Microsoft assured the European Commission (“EC”) during its antitrust review of the ZeniMax purchase that Microsoft would not have the incentive to withhold ZeniMax titles from rival consoles. But, shortly after the EC cleared the transaction, Microsoft made public its decision to make several of the newly acquired ZeniMax titles, including Starfield, Redfall, and Elder Scrolls VI, Microsoft exclusives.

View this comment - View article - View full comments

Quoting: Purple Library Guy

Quoting: massatt212i dont like it, they should allow Microsoft to acquire Activision, Sony is steady buying up single studios and everyone is worshipping it

I must have missed the existence of Sony churches. Really, I've never heard anyone say much good about Sony . . . does that actually happen?

Quoting: massatt212i dont like it, they should allow Microsoft to acquire Activision, Sony is steady buying up single studios and everyone is worshipping it, now Microsoft wants to compete its a problem, its so weird, i believe Blizzard will be in better hands with Microsoft compare to Activision.

Everyone saying microsoft if a Terrible company, just remember what Micheal Jackson said about sony, they both are bad.

This is not about which company is good or bad, this is about a merger having the potential to create a monopoly or a near monopoly in the market. So it also have zero to do with the amount of studios being bought (still MS have bought up more studios then Sony), instead it have to do with the huge size of Activision Blizzard, that is the single thing that activated FTC and why they are trying to block it. It's the size, nothing more.

View this comment - View article - View full comments

Quoting: sarmadWhat's with the "Sexual Content" label? 🤔

It's the Bible, there's rape and sex with minors in it.

View this comment - View article - View full comments

I like this project more: BBC: Youths reveal racy Bible calendar [External Link] :)

View this comment - View article - View full comments

Quoting: Purple Library GuyIt's nice they optimized rats. Wouldn't want to be attacked by plague rats that were sub-optimal.

It's almost as important as the rats being low!

View this comment - View article - View full comments