Why ext3 doesn't need defragmentation ?

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Why ext3 doesn't need defragmentation ?

Bruno Costacurta-7
Hello,
it appears that ext3 doesn't need a real defragmentation operation (by 'real'
I mean a specific tool that need to be run sometimes related to disk usage).

Is it correct ?
If so how it works ?
Related to ext3 standard setup (see tune2fs) the disk is check by default
every 30 boots or 30 days. Is this operation includeing some defragmentation ?

Note: by the way I think the filesystem present in Vista still need to be
defrag...well, once again a feature that was announced but not delivered by
Bill (please do not hesitate to correct me on these assumptions as I'm not a
user of such $$$ product).

Thanks for your attention.
Bye,
Bruno


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Why ext3 doesn't need defragmentation ?

Kamaraju S Kusumanchi
On Sat, 17 Nov 2007 13:46:56 +0100, Bruno Costacurta wrote:

> Hello,
> it appears that ext3 doesn't need a real defragmentation operation (by
> 'real' I mean a specific tool that need to be run sometimes related to
> disk usage).
>
> Is it correct ?
> If so how it works ?

I am not a file system expert by any means. Sometime back I read the
following link and found that it explains this issue very clearly in
layman terms.

http://www.whylinuxisbetter.net/items/defragment/index.php


hth
raju


--
Kamaraju S Kusumanchi
http://www.people.cornell.edu/pages/kk288/
http://malayamaarutham.blogspot.com/


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Why ext3 doesn't need defragmentation ?

Sam-102
Hi Kamaraju,

I am also not an expert in filesystems, however, I do know that the new native linux fs ext3 has what is called "journaling". Linux keeps track of where everything goes, so it doesnt get disorganized. Once in a while you could run fsck if your really concerned about it.

On 11/17/07, Kamaraju Kusumanchi <[hidden email]> wrote:
On Sat, 17 Nov 2007 13:46:56 +0100, Bruno Costacurta wrote:

> Hello,
> it appears that ext3 doesn't need a real defragmentation operation (by
> 'real' I mean a specific tool that need to be run sometimes related to
> disk usage).
>
> Is it correct ?
> If so how it works ?

I am not a file system expert by any means. Sometime back I read the
following link and found that it explains this issue very clearly in
layman terms.

http://www.whylinuxisbetter.net/items/defragment/index.php


hth
raju


--
Kamaraju S Kusumanchi
http://www.people.cornell.edu/pages/kk288/
http://malayamaarutham.blogspot.com/


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Why ext3 doesn't need defragmentation ?

Ron Johnson
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/17/07 22:10, Sam wrote:
> Hi Kamaraju,
>
> I am also not an expert in filesystems, however, I do know that the new
> native linux fs ext3 has what is called "journaling". Linux keeps track
> of where everything goes, so it doesnt get disorganized.

That's *not* the purpose nor a function of journaling.  To say so
indicates a deep misunderstanding of it's nature.

>                                                          Once in a while
> you could run fsck if your really concerned about it.
>
> On 11/17/07, *Kamaraju Kusumanchi* <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     On Sat, 17 Nov 2007 13:46:56 +0100, Bruno Costacurta wrote:
>
>     > Hello,
>     > it appears that ext3 doesn't need a real defragmentation operation (by
>     > 'real' I mean a specific tool that need to be run sometimes
>     related to
>     > disk usage).
>     >
>     > Is it correct ?
>     > If so how it works ?
>
>     I am not a file system expert by any means. Sometime back I read the
>     following link and found that it explains this issue very clearly in
>     layman terms.
>
>     http://www.whylinuxisbetter.net/items/defragment/index.php
>
>
>     hth
>     raju



- --
Ron Johnson, Jr.
Jefferson LA  USA

%SYSTEM-F-FISH, my hovercraft is full of eels
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHP7cUS9HxQb37XmcRArGrAKCs7irEQI+ZXLY/wAn5l2H355Ey4ACfTZSa
lZScvz2dO0DkpQ1B0Sklm0k=
=w+oB
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Why ext3 doesn't need defragmentation ?

Paul Johnson
In reply to this post by Bruno Costacurta-7
On Nov 17, 5:00 am, Bruno Costacurta <[hidden email]> wrote:
> Hello,
> it appears that ext3 doesn't need a real defragmentation operation (by 'real'
> I mean a specific tool that need to be run sometimes related to disk usage).
>
> Is it correct ?

Right.

> If so how it works ?

It shoots for blocks most able to handle the entire data chunk instead
of shooting for the first available chunk no matter what like every
Microsoft filesystem does.  You'd think MS would bring their
filesystem design out of the 1970s already...

> Related to ext3 standard setup (see tune2fs) the disk is check by default
> every 30 boots or 30 days. Is this operation includeing some defragmentation ?

No.  Defragmenting filesystems that don't suffer from it appreciably
to begin with is a great way to needlessly endanger data integrity
with no benefit.


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Why ext3 doesn't need defragmentation ?

David Brodbeck-3

On Nov 18, 2007, at 10:57 AM, Paul Johnson wrote:

> On Nov 17, 5:00 am, Bruno Costacurta <[hidden email]> wrote:
>> Hello,
>> it appears that ext3 doesn't need a real defragmentation operation  
>> (by 'real'
>> I mean a specific tool that need to be run sometimes related to  
>> disk usage).
>>
>> Is it correct ?
>
> Right.
>
>> If so how it works ?
>
> It shoots for blocks most able to handle the entire data chunk instead
> of shooting for the first available chunk no matter what like every
> Microsoft filesystem does.

Keep in mind, though, that this will only prevent fragmentation if  
there's a sufficient amount of free space on the disk.  If you  
constantly keep the disk close to full you will still get  
fragmentation, just because there aren't many free space chunks to  
shoot for.




--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Why ext3 doesn't need defragmentation ?

Stefan Monnier
In reply to this post by Bruno Costacurta-7
> Note: by the way I think the filesystem present in Vista still need to be
> defrag...well, once again a feature that was announced but not delivered by
> Bill (please do not hesitate to correct me on these assumptions as I'm not a
> user of such $$$ product).

No.  The NTFS file system does not need defragmentation.
The best explanation I've heard of why they have a defragmenter for it
is that it was considered easier to write a defragmenter than to go
about explaining that FAT32 just sucks.


        Stefan


PS: The only time I ran XP's defragmenter for its NTFS filesystem (so as
to make space for Debian using ntfsresize back when it was not yet able
to move files on its own), my wife subsequently complained that her
machine booted (and generally felt) slower ;-)


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Why ext3 doesn't need defragmentation ?

David Brodbeck-3

On Nov 30, 2007, at 9:45 AM, Stefan Monnier wrote:
> No.  The NTFS file system does not need defragmentation.
> The best explanation I've heard of why they have a defragmenter for it
> is that it was considered easier to write a defragmenter than to go
> about explaining that FAT32 just sucks.

Are you sure about that?  I've seen some really heavily fragmented  
NTFS filesystems.  Or are you saying that fragmentation doesn't  
affect NTFS's performance?

Microsoft didn't provide a defragmenter for NTFS until Windows 2000  
-- on NT4 and earlier they suggested you buy a third-party utility.


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Why ext3 doesn't need defragmentation ?

Alvin Oga

hi ya

> David Brodbeck wrote:
>
> On Nov 30, 2007, at 9:45 AM, Stefan Monnier wrote:
> > No.  The NTFS file system does not need defragmentation.

all file systems can use a defragmentor

lets assume a disk format of:

1 2 3 4 5 6 7 8 9 10 ...... 63    ( aka sectors )

if you try to read/write a file starting sector 1 into your disk buffer,
you may or may not have disk buffer space left to read another 512Byte
from sector 2...

if you wait a bit, few milliseconds for the system to services its
disk interrupts, you now have disk buffer space to read sector 2 ..

but since you waited too long, sector 2 came and went, so now you have
to wait for a whole revolution before you can read sector 2

----

if you format using, than you may or may not have time to read sector 2

1 11 21 31 41 51 61 2 12 22 32 42 52 62 3 13 23 33 43 53 63 4 14...
 
the defragmentor can be used to move sectors around to optimize
reading the whole file w/o waiting for the next revolution

        - how the defragmentor displays used and unused sectors
        can make a big difference in the pretty pic you see vs the
        actual performance

what you see the defragmentor showing would be a continuously
allocated file instead of scattered across various sectors
within a track or having to move the heads to a different tract
to get to the next 512byte

there's only 512bytes per sector
63 sectors per track
and any number of cylinders depending on your disk size

the number of heads and disk buffer size would depend on your
disk drive manufacturer and model#

one traack is 512MB * 62 == 31.744KBytes
with 16 physical heads .... you can read 509.904KBytes per revolution

all un-used disk sectors belonging to a different file is read and
discarded ... what a waste

with 8MB or 16MB disk buffer .. you can read lots of tracks before
the disk buffer is full ...  there should NOT be a "slow" system

lba ...
maps all the cylinder/heads/sector into other whacky numbers ( lba blocks )

c ya
alvin


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Why ext3 doesn't need defragmentation ?

Sam Leon-2
Alvin Oga wrote:
 > hi ya
 >
 >> David Brodbeck wrote:
 >>
 >> On Nov 30, 2007, at 9:45 AM, Stefan Monnier wrote:
 >>> No.  The NTFS file system does not need defragmentation.
 >
 > all file systems can use a defragmentor
 >
 > lets assume a disk format of:
 >
 > 1 2 3 4 5 6 7 8 9 10 ...... 63    ( aka sectors )
 >
 > if you try to read/write a file starting sector 1 into your disk buffer,
 > you may or may not have disk buffer space left to read another 512Byte
 > from sector 2...
 >
 > if you wait a bit, few milliseconds for the system to services its
 > disk interrupts, you now have disk buffer space to read sector 2 ..
 >
 > but since you waited too long, sector 2 came and went, so now you have
 > to wait for a whole revolution before you can read sector 2
 >
 > ----
 >
 > if you format using, than you may or may not have time to read sector 2
 >
 > 1 11 21 31 41 51 61 2 12 22 32 42 52 62 3 13 23 33 43 53 63 4 14...
 >
 > the defragmentor can be used to move sectors around to optimize
 > reading the whole file w/o waiting for the next revolution
 >
 > - how the defragmentor displays used and unused sectors
 > can make a big difference in the pretty pic you see vs the
 > actual performance
 >
 > what you see the defragmentor showing would be a continuously
 > allocated file instead of scattered across various sectors
 > within a track or having to move the heads to a different tract
 > to get to the next 512byte
 >
 > there's only 512bytes per sector
 > 63 sectors per track
 > and any number of cylinders depending on your disk size
 >
 > the number of heads and disk buffer size would depend on your
 > disk drive manufacturer and model#
 >
 > one traack is 512MB * 62 == 31.744KBytes
 > with 16 physical heads .... you can read 509.904KBytes per revolution
 >
 > all un-used disk sectors belonging to a different file is read and
 > discarded ... what a waste
 >
 > with 8MB or 16MB disk buffer .. you can read lots of tracks before
 > the disk buffer is full ...  there should NOT be a "slow" system
 >
 > lba ...
 > maps all the cylinder/heads/sector into other whacky numbers ( lba
blocks )
 >
 > c ya
 > alvin
 >
 >


Great info thank you!

Sam


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Email etiquette (was Re: Why ext3 doesn't need defragmentation ?)

Ron Johnson
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/30/07 20:58, Sam Leon wrote:

> Alvin Oga wrote:
>> hi ya
>>
>>> David Brodbeck wrote:
>>>
>>> On Nov 30, 2007, at 9:45 AM, Stefan Monnier wrote:
>>>> No.  The NTFS file system does not need defragmentation.
>>
>> all file systems can use a defragmentor
>>
[snip 47 lines]

>>
>> lba ...
>> maps all the cylinder/heads/sector into other whacky numbers ( lba
> blocks )
>>
>> c ya
>> alvin
>>
>>
>
>
> Great info thank you!

Repeating a whole, long email just just for a 4 word reply is why
top-posters don't see the value of in-line posting.

- --
Ron Johnson, Jr.
Jefferson LA  USA

%SYSTEM-F-FISH, my hovercraft is full of eels
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHUNp+S9HxQb37XmcRArljAJ40l9z6bAbl0s6Fm7Dmv4KlRh3uhgCgoKtJ
GCJH8LVIQiy0ZB+LPnv0Tnw=
=g+Ng
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Why ext3 doesn't need defragmentation ?

Rob Sims-5
In reply to this post by Alvin Oga
On Fri, Nov 30, 2007 at 02:53:32PM -0800, Alvin Oga wrote:
> > David Brodbeck wrote:
> >
> > On Nov 30, 2007, at 9:45 AM, Stefan Monnier wrote:
> > > No.  The NTFS file system does not need defragmentation.
>
> all file systems can use a defragmentor

...description of interleaving deleted...

> the defragmentor can be used to move sectors around to optimize
> reading the whole file w/o waiting for the next revolution

This is called interleaving, and has nothing to do with fragmentation.
Defragmentors do nothing about interleave.  If one were to interleave at
the filesystem level, performance would quickly get poor because of the
huge number of pointers to chunks of the file.

Interleaving is done at the hardware format level, and because the
controller is integrated with the media, is completely hidden from the
host.  Note that in a sequential read case, performance would be half or
worse for interleaved sectors, as you'd suffer through at least two
rotations to read each track.

> - how the defragmentor displays used and unused sectors
> can make a big difference in the pretty pic you see vs the
> actual performance

> what you see the defragmentor showing would be a continuously
> allocated file instead of scattered across various sectors
> within a track or having to move the heads to a different tract
> to get to the next 512byte

> there's only 512bytes per sector
> 63 sectors per track
> and any number of cylinders depending on your disk size

I'll bet you can't find a hard drive made in the past decade with 63
sectors per track.  Hard drives have a number that varies with the
circumference of the track.  The 63 spt is an artificial construct for
backwards compatibility.

> lba ...
> maps all the cylinder/heads/sector into other whacky numbers ( lba blocks )

A continuous range from 0 to (# sectors - 1) is whacky?  Compared to
using physical c/h/s numbers that bear no relation to reality?  
 

--
Rob

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Why ext3 doesn't need defragmentation ?

Stefan Monnier
In reply to this post by David Brodbeck-3
> Are you sure about that?  I've seen some really heavily fragmented NTFS
> filesystems.

Sure, fragmentation does occur, just like in any other filesystem.

> Or are you saying that fragmentation doesn't  affect
> NTFS's performance?

Yup.  You can of course build an artificial worst-case test that will
cause serious performance penalties.  But that is irrelevant to
normal use.


        Stefan


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]