Quoting: Guest

Quoting: F.Ultra

Quoting: Guest

Btrfs file system performance improvements.

Is long mounting of large HDD partitions fixed now?

What counts as large partitions and long time? I use BTRFS on several servers each with 153TB per partition and mounting is sub second and have been for many years.

edit: that said one of the listed items is improved mount times on large systems:

Hi,

please pull the following updates for btrfs. There's a bunch of
performance improvements, most notably the FIEMAP speedup, the new block
group tree to speed up mount on large filesystems, more io_uring
integration, some sysfs exports and the usual fixes and core updates.

Thanks.

---

Performance:

- outstanding FIEMAP speed improvement
- algorithmic change how extents are enumerated leads to orders of
magnitude speed boost (uncached and cached)
- extent sharing check speedup (2.2x uncached, 3x cached)
- add more cancellation points, allowing to interrupt seeking in files
with large number of extents
- more efficient hole and data seeking (4x uncached, 1.3x cached)
- sample results:
256M, 32K extents: 4s -> 29ms (~150x)
512M, 64K extents: 30s -> 59ms (~550x)
1G, 128K extents: 225s -> 120ms (~1800x)

- improved inode logging, especially for directories (on dbench workload
throughput +25%, max latency -21%)

- improved buffered IO, remove redundant extent state tracking, lowering
memory consumption and avoiding rb tree traversal

- add sysfs tunable to let qgroup temporarily skip exact accounting when
deleting snapshot, leading to a speedup but requiring a rescan after
that, will be used by snapper

- support io_uring and buffered writes, until now it was just for direct
IO, with the no-wait semantics implemented in the buffered write path
it now works and leads to speed improvement in IOPS (2x), throughput
(2.2x), latency (depends, 2x to 150x)

- small performance improvements when dropping and searching for extent
maps as well as when flushing delalloc in COW mode (throughput +5MB/s)

User visible changes:

- new incompatible feature block-group-tree adding a dedicated tree for
tracking block groups, this allows a much faster load during mount and
avoids seeking unlike when it's scattered in the extent tree items
- this reduces mount time for many-terabyte sized filesystems
- conversion tool will be provided so existing filesystem can also be
updated in place
- to reduce test matrix and feature combinations requires no-holes
and free-space-tree (mkfs defaults since 5.15)

- improved reporting of super block corruption detected by scrub

- scrub also tries to repair super block and does not wait until next
commit

- discard stats and tunables are exported in sysfs
(/sys/fs/btrfs/FSID/discard)

- qgroup status is exported in sysfs (/sys/sys/fs/btrfs/FSID/qgroups/)

- verify that super block was not modified when thawing filesystem

Fixes:

- FIEMAP fixes
- fix extent sharing status, does not depend on the cached status where
merged
- flush delalloc so compressed extents are reported correctly

- fix alignment of VMA for memory mapped files on THP

- send: fix failures when processing inodes with no links (orphan files
and directories)

- fix race between quota enable and quota rescan ioctl

- handle more corner cases for read-only compat feature verification

- fix missed extent on fsync after dropping extent maps

Core:

- lockdep annotations to validate various transactions states and state
transitions

- preliminary support for fs-verity in send

- more effective memory use in scrub for subpage where sector is smaller
than page

- block group caching progress logic has been removed, load is now
synchronous

- simplify end IO callbacks and bio handling, use chained bios instead
of own tracking

- add no-wait semantics to several functions (tree search, nocow,
flushing, buffered write

- cleanups and refactoring

MM changes:

- export balance_dirty_pages_ratelimited_flags

known issue for HDD (rotational disks)
https://bbs.archlinux.org/viewtopic.php?id=272376 [External Link]
https://lore.kernel.org/linux-btrfs/CAMQzBqCSzr4UO1VFTjtSDPt+0ukhf6yqK=q+eLA+Tp1hiB_weA@mail.gmail.com/t/#u [External Link]
https://lore.kernel.org/linux-btrfs/CAHQ7scVGPAwEGQOq3Kmn75GJzyzSQ9qrBBZrHFu+4YWQhGE0Lw@mail.gmail.com/t/#u [External Link]
That guy has 16TB disk and it takes under 1 minute to mount it.
I have 3.2TB partition and it mounts in 17 seconds, with defragmented extents.

Well I use rotational HDD:s as well (so ok they are SAS and not SATA drives but that shouldn't matter), in the latest machine I configured it was 36 x Seagate Exos X18 3.5 HDD 16TB SAS and it mounts almost instantly.

edit: not saying that it doesn't happen to people because it obviously are, just that it does not seem to be universal. Anyway here is to hoping that the changes in 6.1 solves it for the people that are experiencing this issue.

View this comment - View article - View full comments

Quoting: pleasereadthemanualEdit: And if you're familiar with the fable of Microsoft v Netscape, there's a monopoly that ended well for customers everywhere. Thanks to Microsoft, they standardized the idea of browsers everywhere being free-of-charge, which forced Netscape to pivot into Mozilla, which resulted in a great free software browser (that was also free-of-charge) that subsequently took market share from Internet Explorer because it was a better browser that supported open standards (some of which it had invented itself, like JavaScript) better than the competition.

If Netscape won, what would the world look like? Microsoft would have had to charge for Internet Explorer instead of bundling it with Windows 95 and future editions. Netscape wouldn't have been forced to innovate. Mozilla Firefox wouldn't exist, nor would any of Mozilla's other great free software programs.

The IE monopoly had zero to do with if a browser should be free or not, browsers had been free to download and use for some years before Microsoft decided to bundle IE with Windows and thus using their monopoly on the OS to gain leverage in the browser market. Nor was "give anyone a free browser" the aim of Microsoft, their goal was to shift enterprises to use their web server ISS instead of the Netscape Web Server (and in the end they both lost out to Apache anyway).

The problem was never that MS developed IE or that they released it for free, it was all about them abusing their monopoly. Fully agree that we probably in the end got it better with Firefox than if Netscape Navigator had survived (although we will never know since we don't have access to that parallel universe) but that is a "despite of" and not "thanks to" the market abuse by MS.

View this comment - View article - View full comments

Quoting: Guest

Btrfs file system performance improvements.

Is long mounting of large HDD partitions fixed now?

What counts as large partitions and long time? I use BTRFS on several servers each with 153TB per partition and mounting is sub second and have been for many years.

edit: that said one of the listed items is improved mount times on large systems:

Hi,

please pull the following updates for btrfs. There's a bunch of
performance improvements, most notably the FIEMAP speedup, the new block
group tree to speed up mount on large filesystems, more io_uring
integration, some sysfs exports and the usual fixes and core updates.

Thanks.

---

Performance:

- outstanding FIEMAP speed improvement
- algorithmic change how extents are enumerated leads to orders of
magnitude speed boost (uncached and cached)
- extent sharing check speedup (2.2x uncached, 3x cached)
- add more cancellation points, allowing to interrupt seeking in files
with large number of extents
- more efficient hole and data seeking (4x uncached, 1.3x cached)
- sample results:
256M, 32K extents: 4s -> 29ms (~150x)
512M, 64K extents: 30s -> 59ms (~550x)
1G, 128K extents: 225s -> 120ms (~1800x)

- improved inode logging, especially for directories (on dbench workload
throughput +25%, max latency -21%)

- improved buffered IO, remove redundant extent state tracking, lowering
memory consumption and avoiding rb tree traversal

- add sysfs tunable to let qgroup temporarily skip exact accounting when
deleting snapshot, leading to a speedup but requiring a rescan after
that, will be used by snapper

- support io_uring and buffered writes, until now it was just for direct
IO, with the no-wait semantics implemented in the buffered write path
it now works and leads to speed improvement in IOPS (2x), throughput
(2.2x), latency (depends, 2x to 150x)

- small performance improvements when dropping and searching for extent
maps as well as when flushing delalloc in COW mode (throughput +5MB/s)

User visible changes:

- new incompatible feature block-group-tree adding a dedicated tree for
tracking block groups, this allows a much faster load during mount and
avoids seeking unlike when it's scattered in the extent tree items
- this reduces mount time for many-terabyte sized filesystems
- conversion tool will be provided so existing filesystem can also be
updated in place
- to reduce test matrix and feature combinations requires no-holes
and free-space-tree (mkfs defaults since 5.15)

- improved reporting of super block corruption detected by scrub

- scrub also tries to repair super block and does not wait until next
commit

- discard stats and tunables are exported in sysfs
(/sys/fs/btrfs/FSID/discard)

- qgroup status is exported in sysfs (/sys/sys/fs/btrfs/FSID/qgroups/)

- verify that super block was not modified when thawing filesystem

Fixes:

- FIEMAP fixes
- fix extent sharing status, does not depend on the cached status where
merged
- flush delalloc so compressed extents are reported correctly

- fix alignment of VMA for memory mapped files on THP

- send: fix failures when processing inodes with no links (orphan files
and directories)

- fix race between quota enable and quota rescan ioctl

- handle more corner cases for read-only compat feature verification

- fix missed extent on fsync after dropping extent maps

Core:

- lockdep annotations to validate various transactions states and state
transitions

- preliminary support for fs-verity in send

- more effective memory use in scrub for subpage where sector is smaller
than page

- block group caching progress logic has been removed, load is now
synchronous

- simplify end IO callbacks and bio handling, use chained bios instead
of own tracking

- add no-wait semantics to several functions (tree search, nocow,
flushing, buffered write

- cleanups and refactoring

MM changes:

- export balance_dirty_pages_ratelimited_flags

View this comment - View article - View full comments

Quoting: massatt212

Quoting: F.Ultra

Quoting: Purple Library Guy

Quoting: massatt212i dont like it, they should allow Microsoft to acquire Activision, Sony is steady buying up single studios and everyone is worshipping it

I must have missed the existence of Sony churches. Really, I've never heard anyone say much good about Sony . . . does that actually happen?

Quoting: massatt212i dont like it, they should allow Microsoft to acquire Activision, Sony is steady buying up single studios and everyone is worshipping it, now Microsoft wants to compete its a problem, its so weird, i believe Blizzard will be in better hands with Microsoft compare to Activision.

Everyone saying microsoft if a Terrible company, just remember what Micheal Jackson said about sony, they both are bad.

This is not about which company is good or bad, this is about a merger having the potential to create a monopoly or a near monopoly in the market. So it also have zero to do with the amount of studios being bought (still MS have bought up more studios then Sony), instead it have to do with the huge size of Activision Blizzard, that is the single thing that activated FTC and why they are trying to block it. It's the size, nothing more.

No its not the size of Activision blizzard they are fighting about, its about the Call of duty Franchise, they dont care about the Blizzard games, its all about COD, cause if microsoft leaves says they dont want COD Sony wont have a problem with the acquirement.

You are talking about why SONY is concerned about the merger, I was talking about the FTC. The size of the deal is what made the governmental regulators like The FTC to start their investigation and they did this long before SONY said anything.

SOMY might be completely fine with the deal if COD is left out of the deal, The FTC will not. Key to this all is point #96 in their suit:

96. The Proposed Acquisition is reasonably likely to substantially lessen competition or tend to create a monopoly in the Relevant Markets by creating a combined firm with the ability and increased incentive to withhold Activision’s valuable gaming content from, or degrade Activision’s content for, Microsoft’s rivals. The combined firm would have the ability to exclude Microsoft’s rivals from access to some or all of Activision’s content in the Relevant Markets. Microsoft would have the power to decide if, when, and to what extent Activision content will be available on competing products. The Proposed Acquisition is likely to increase entry barriers, thereby dampening beneficial rivalry and innovation. If permitted to make Activision a captive supplier, Microsoft would have a substantially increased incentive to engage in strategies to that would likely lead to reduced consumer choice, higher prices or lower quality products, and less innovation.

Especially damning for MS is point #12 however as I understand it the EU commission have come out and clarified that they think that the FTC have gotten this wrong in that the EC allowed the deal without such a promise:

12. Microsoft’s past conduct provides a preview of the combined firm’s likely plans if it consummates the Proposed Acquisition, despite any assurances the company may offer regarding its plans. In March 2021, Microsoft acquired ZeniMax Media Inc. (“ZeniMax”), the parent company of the well-known game developer and publisher Bethesda Softworks LLC (“Bethesda”). Microsoft assured the European Commission (“EC”) during its antitrust review of the ZeniMax purchase that Microsoft would not have the incentive to withhold ZeniMax titles from rival consoles. But, shortly after the EC cleared the transaction, Microsoft made public its decision to make several of the newly acquired ZeniMax titles, including Starfield, Redfall, and Elder Scrolls VI, Microsoft exclusives.

View this comment - View article - View full comments

Quoting: Purple Library Guy

Quoting: massatt212i dont like it, they should allow Microsoft to acquire Activision, Sony is steady buying up single studios and everyone is worshipping it

I must have missed the existence of Sony churches. Really, I've never heard anyone say much good about Sony . . . does that actually happen?

Quoting: massatt212i dont like it, they should allow Microsoft to acquire Activision, Sony is steady buying up single studios and everyone is worshipping it, now Microsoft wants to compete its a problem, its so weird, i believe Blizzard will be in better hands with Microsoft compare to Activision.

Everyone saying microsoft if a Terrible company, just remember what Micheal Jackson said about sony, they both are bad.

This is not about which company is good or bad, this is about a merger having the potential to create a monopoly or a near monopoly in the market. So it also have zero to do with the amount of studios being bought (still MS have bought up more studios then Sony), instead it have to do with the huge size of Activision Blizzard, that is the single thing that activated FTC and why they are trying to block it. It's the size, nothing more.

View this comment - View article - View full comments

Quoting: sarmadWhat's with the "Sexual Content" label? 🤔

It's the Bible, there's rape and sex with minors in it.

View this comment - View article - View full comments

I like this project more: BBC: Youths reveal racy Bible calendar [External Link] :)

View this comment - View article - View full comments

Quoting: Purple Library GuyIt's nice they optimized rats. Wouldn't want to be attacked by plague rats that were sub-optimal.

It's almost as important as the rats being low!

View this comment - View article - View full comments

Quoting: Valck

Quoting: raptor85

Quoting: ArehandoroAm I the only one to think that even AMD's new cards are better than Nvidia's on power consumption, 300/355 TDP is still crazy high?

With climate change creeping up on us, current international political affairs affecting prices and availability of energy, and the market saying yes to devices like the Switch and the Deck, perhaps it's about time to demand these companies to release things more align with what the world needs.

We need to use much less energy. Not the same, and definitely not more.

There's still 100 watt and less cards on the market widely available, nobody's putting a gun to your head forcing you to get a higher end card. An extra 200 watts is also not really a lot compared to most other things in a normal household, you can literally offset that usage by not watching TV while the computer is on.

Averaged over the past year and rounded up generously, my entire single person household averaged 160 Watts. And I am confident I could reduce that by another 25% without too much effort.
Peak usage is undeniably higher when the microwave/tea kettle/washing machine are on, but they aren't for most of the time. Top consumers are the PC and the fridge, with the PC sitting at a whopping 80 right now, and hitting 200 with heavy gaming which I admittedly don't do that much these days, most of the games get by on 150ish or so.
I don't play the likes of CP77 or twitchy shooter type games, I had my fill of that crap when I was younger, but some of the sims I like do look pretty and have their respective cost in wall power.

So in summary, even "just" two hundred watts just for a GPU is insane, as are a thousand credits to pay for it, and nobody needs to commend AMD or Nvidia for not going higher still.
Consumers and marketing people alike need to get off that trainSUV, and fast. In fact, DO take the train the next time.

ED: To be fair, that number doesn't include heating, which is still fossil gas and would probably amount to about the same 150ish.figure again, and to bring that down in any meaningful way would require a massive investment in insulation on the landlord's side. Which I'm fairly certain will be coming, if and when legislation eventually requires it. So we're talking 350ish Watts instead, but still. That's one entire household's worth, just for the GPU.

On the other hand the card uses 300watts on peak, not during idle. My RX480 is rated to draw 100W but currently browsing here with a YT video playing in the background it draws about 7-10W and it's very likely that the RX7900XTX would draw even less under the same load.

View this comment - View article - View full comments

Quoting: Shmerl

Quoting: F.UltraIt's for people doing machine learning, they use GPU:s to handle the vast amounts of data so AMD, nVidia (and now Intel) are implementing new functions on their cards to improve the performance in that area. Nothing that helps the rest of us who only use these for displaying graphics and playing games.

Nothing stops games from using AI you know, for more realistic behaviors and simulation, not for graphics :)

It feels like a lot of games are running after better graphics but really almost no one is trying to improve world simulation quality.

True, I don't know if any of that game AI uses any of the ML functionality of the GPU:s yet though.

edit: AFAIK also those ML extensions have to do with the learning part of the AI research and not the end result of said research but then again I have basically zero knowledge of ML so should just shut up :)

View this comment - View article - View full comments