How to make apps play nice?

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

How to make apps play nice?

Craig Hagerman
I'm running Debian amd64 on an Asus board with an AMD64 3200+ and a gig of ram.

This box is mainly used as a home media center and file server. Most
of the time the system monitor shows that CPU usage is around 23% and
ram usage between 40% and 80%. But for some reason when I copy files
(cp or scp network copying) it seems to take over - CPU usage shoots
up to 100% and everything else starts stuttering. Same thing also
happens with running find command or other programs. If I am listening
to music it starts skipping, if I am watching video, it basically
stops. Why is this? I know I can use 'nice' to change the niceness of
a program, but I want to know how to alter settings so that things
like cp, scp, find, etc do not hog up the entire CPU all the time, not
just on a case-by-case basis.

Any help or advice would be appreciated.

Craig

Reply | Threaded
Open this post in threaded view
|

Re: How to make apps play nice?

Christoph Fassbach
Hi!

You did not forget to activate dma for your drives, did you?

Christoph

Craig Hagerman wrote:

>I'm running Debian amd64 on an Asus board with an AMD64 3200+ and a gig of ram.
>
>This box is mainly used as a home media center and file server. Most
>of the time the system monitor shows that CPU usage is around 23% and
>ram usage between 40% and 80%. But for some reason when I copy files
>(cp or scp network copying) it seems to take over - CPU usage shoots
>up to 100% and everything else starts stuttering. Same thing also
>happens with running find command or other programs. If I am listening
>to music it starts skipping, if I am watching video, it basically
>stops. Why is this? I know I can use 'nice' to change the niceness of
>a program, but I want to know how to alter settings so that things
>like cp, scp, find, etc do not hog up the entire CPU all the time, not
>just on a case-by-case basis.
>
>Any help or advice would be appreciated.
>
>Craig
>
>
>  
>


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to make apps play nice?

Jean-Christophe Montigny
In reply to this post by Craig Hagerman
Hello,

Use top or htop to see what is exactly using the CPU.
23% CPU usage is odd if the machine is doing nothing.
Check if it's system that is using your whole CPU.
Generally speaking, anyway, it depends on disk usage, i mean, doing a
scp over a 1GB connection (local) will have a large toll on the system
and that's why we have SCSI that's much better armed to face that kind
of transfer. Even 100Mbits can make your hd work hard... very hard. And
there's not much you can do there.

The other person is right. If you see that 100% of the CPU is being used
by system, either your hard drive controller is not optimally configured
(see read/write cache, dma, and stuff), or it's crap, or consider
switching to scsi :) Anyway, *nix systems don't do "find" all the time :)

Craig Hagerman wrote:

> I'm running Debian amd64 on an Asus board with an AMD64 3200+ and a gig of ram.
>
> This box is mainly used as a home media center and file server. Most
> of the time the system monitor shows that CPU usage is around 23% and
> ram usage between 40% and 80%. But for some reason when I copy files
> (cp or scp network copying) it seems to take over - CPU usage shoots
> up to 100% and everything else starts stuttering. Same thing also
> happens with running find command or other programs. If I am listening
> to music it starts skipping, if I am watching video, it basically
> stops. Why is this? I know I can use 'nice' to change the niceness of
> a program, but I want to know how to alter settings so that things
> like cp, scp, find, etc do not hog up the entire CPU all the time, not
> just on a case-by-case basis.
>
> Any help or advice would be appreciated.
>
> Craig
>

--
Jean-Christophe Montigny
Etudiant de troisième année à Grenoble Ecole de Management
Actuellement en parcours ingénieur à l'Ecole Nationale Supérieure de
Télécommunications de Bretagne

jcm.vcf (335 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

(follow) Re: How to make apps play nice?

Jean-Christophe Montigny
In reply to this post by Craig Hagerman
Re,

Oh and i forgot something much simpler.
If you have multiple disks, just put your system on one, and the files
you regularly tranfer on another. That way, you can have one harddrive
beinbg 100% used without impacting on your video.
Btw, if you hate find (hey... except if you use your computer while
watching movies, there's no reason for find to run while you're wathcing
a video, except the cron, and, ... well, it runs like, at 6am ? ;) )
you can tell the cronjob not to go to some locations... like to your
second harddrive :)

see /etc/updatedb.conf for that

Craig Hagerman wrote:

> I'm running Debian amd64 on an Asus board with an AMD64 3200+ and a gig of ram.
>
> This box is mainly used as a home media center and file server. Most
> of the time the system monitor shows that CPU usage is around 23% and
> ram usage between 40% and 80%. But for some reason when I copy files
> (cp or scp network copying) it seems to take over - CPU usage shoots
> up to 100% and everything else starts stuttering. Same thing also
> happens with running find command or other programs. If I am listening
> to music it starts skipping, if I am watching video, it basically
> stops. Why is this? I know I can use 'nice' to change the niceness of
> a program, but I want to know how to alter settings so that things
> like cp, scp, find, etc do not hog up the entire CPU all the time, not
> just on a case-by-case basis.
>
> Any help or advice would be appreciated.
>
> Craig
>

--
Jean-Christophe Montigny
Etudiant de troisième année à Grenoble Ecole de Management
Actuellement en parcours ingénieur à l'Ecole Nationale Supérieure de
Télécommunications de Bretagne

jcm.vcf (335 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: How to make apps play nice?

Torben Keil
In reply to this post by Craig Hagerman
Hi,

you have a cheap network card and the driver for it needs the wohle CPU, do
you?

Greets,
Torben

On Saturday 17 December 2005 01:58, Craig Hagerman wrote:

> I'm running Debian amd64 on an Asus board with an AMD64 3200+ and a gig of
> ram.
>
> This box is mainly used as a home media center and file server. Most
> of the time the system monitor shows that CPU usage is around 23% and
> ram usage between 40% and 80%. But for some reason when I copy files
> (cp or scp network copying) it seems to take over - CPU usage shoots
> up to 100% and everything else starts stuttering. Same thing also
> happens with running find command or other programs. If I am listening
> to music it starts skipping, if I am watching video, it basically
> stops. Why is this? I know I can use 'nice' to change the niceness of
> a program, but I want to know how to alter settings so that things
> like cp, scp, find, etc do not hog up the entire CPU all the time, not
> just on a case-by-case basis.
>
> Any help or advice would be appreciated.
>
> Craig


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to make apps play nice?

zzz haha
i experience the same symptom. which is that i can't enable dma for my
via board. it can be seen with 'hdparm -d1 /dev/hda'

z

> On Saturday 17 December 2005 01:58, Craig Hagerman wrote:
> > I'm running Debian amd64 on an Asus board with an AMD64 3200+ and a gig of
> > ram.
> >
> > This box is mainly used as a home media center and file server. Most
> > of the time the system monitor shows that CPU usage is around 23% and
> > ram usage between 40% and 80%. But for some reason when I copy files
> > (cp or scp network copying) it seems to take over - CPU usage shoots
> > up to 100% and everything else starts stuttering. Same thing also
> > happens with running find command or other programs. If I am listening
> > to music it starts skipping, if I am watching video, it basically
> > stops. Why is this? I know I can use 'nice' to change the niceness of
> > a program, but I want to know how to alter settings so that things
> > like cp, scp, find, etc do not hog up the entire CPU all the time, not
> > just on a case-by-case basis.
> >
> > Any help or advice would be appreciated.
> >
> > Craig
>
>
> --
> To UNSUBSCRIBE, email to [hidden email]
> with a subject of "unsubscribe". Trouble? Contact [hidden email]
>
>

Reply | Threaded
Open this post in threaded view
|

Re: How to make apps play nice?

Oleg Gritsinevich
In reply to this post by Craig Hagerman
On Sat, Dec 17, 2005 at 09:58:17AM +0900, Craig Hagerman wrote:
> I'm running Debian amd64 on an Asus board with an AMD64 3200+ and a gig of ram.
[skip]
>
> Any help or advice would be appreciated.
        It looks like DMA problems with your HDD. Show your /var/log/dmesg
and output of:

# hdparm /dev/<your_hdd>
# hdparm -tT /dev/<your_hdd>

--
With best regards, Oleg Gritsinevich


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to make apps play nice?

Craig Hagerman
Thanks for all the replies. Torben, I think my network card is fine.
The problem is the same with using either ethernet or wireless for
internet.


Sorry for my ignorance... what IS dma anyway? It seems this is the
problem. Here is the output from hdparm:

$ hdparm /dev/hda

/dev/hda:
 multcount    = 16 (on)
 IO_support   =  0 (default 16-bit)
 unmaskirq    =  0 (off)
 using_dma    =  0 (off)
 keepsettings =  0 (off)
 readonly     =  0 (off)
 readahead    = 256 (on)
 geometry     = 65535/16/63, sectors = 240121728, start = 0

$ hdparm /dev/sda

/dev/sda:
 IO_support   =  0 (default 16-bit)
 readonly     =  0 (off)
 readahead    = 256 (on)
 geometry     = 24321/255/63, sectors = 390721968, start = 0

$ hdparm /dev/sdb

/dev/sdb:
 IO_support   =  0 (default 16-bit)
 readonly     =  0 (off)
 readahead    = 256 (on)
 geometry     = 30515/255/63, sectors = 490234752, start = 0

(NOTE- sda and sdb are joined up via LVM2 into one big volume)
So what does all of this mean? Seems that dma is not activated for
/dev/hda. Does the above mean that is IS active for /dev/sda and
/dev/sdb?

the -tT flag gives this:

$ hdparm -tT /dev/hda

/dev/hda:
 Timing cached reads:   2652 MB in  2.00 seconds = 1325.92 MB/sec
 Timing buffered disk reads:   12 MB in  3.14 seconds =   3.82 MB/sec

$ hdparm -tT /dev/sda

/dev/sda:
 Timing cached reads:   2732 MB in  2.00 seconds = 1365.91 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate
ioctl for device
 Timing buffered disk reads:  160 MB in  3.03 seconds =  52.84 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate
ioctl for device

$ hdparm -tT /dev/sdb

/dev/sdb:
 Timing cached reads:   2720 MB in  2.00 seconds = 1359.92 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate
ioctl for device
 Timing buffered disk reads:  172 MB in  3.03 seconds =  56.80 MB/sec
HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate
ioctl for device


Meanwhile the -d activating option fails:

$ hdparm -d1 /dev/hda

/dev/hda:
 setting using_dma to 1 (on)
 HDIO_SET_DMA failed: Operation not permitted
 using_dma    =  0 (off)


Any advice concerning this? How does one get dma activated?
(Thanks again.)

Craig

Reply | Threaded
Open this post in threaded view
|

Re: How to make apps play nice?

Lubos Vrbka-2
> Sorry for my ignorance... what IS dma anyway?
iirc, abbreviation for 'direct memory access'. it is a protocol for
transfering data between harddrive and memory not requiring to move the
data through cpu, but directly - therefore it is much faster.

regards,

--
Lubos
_@_"


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to make apps play nice?

Maciej Matysiak
In reply to this post by Craig Hagerman
On the 18th of December 2005 at 15:35, Craig Hagerman <craighagerman#gmail.com> wrote:

> Meanwhile the -d activating option fails:
>
> $ hdparm -d1 /dev/hda
>
> /dev/hda:
>  setting using_dma to 1 (on)
>  HDIO_SET_DMA failed: Operation not permitted
>  using_dma    =  0 (off)
>
> Any advice concerning this? How does one get dma activated?

Try this command as root :) (I guess that "$" is prompt in non-root shell).

 m.m.
--
 use gnus, not guns!


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to make apps play nice?

Craig Hagerman
On 12/18/05, Maciej Matysiak <[hidden email]> wrote:
> Try this command as root :) (I guess that "$" is prompt in non-root shell).
>
>  m.m.

Those commands WERE given as root. (I have custom prompts and use the
# as root.)

Craig

Reply | Threaded
Open this post in threaded view
|

Re: How to make apps play nice?

Lennart Sorensen
In reply to this post by Craig Hagerman
On Sat, Dec 17, 2005 at 09:58:17AM +0900, Craig Hagerman wrote:

> I'm running Debian amd64 on an Asus board with an AMD64 3200+ and a gig of ram.
>
> This box is mainly used as a home media center and file server. Most
> of the time the system monitor shows that CPU usage is around 23% and
> ram usage between 40% and 80%. But for some reason when I copy files
> (cp or scp network copying) it seems to take over - CPU usage shoots
> up to 100% and everything else starts stuttering. Same thing also
> happens with running find command or other programs. If I am listening
> to music it starts skipping, if I am watching video, it basically
> stops. Why is this? I know I can use 'nice' to change the niceness of
> a program, but I want to know how to alter settings so that things
> like cp, scp, find, etc do not hog up the entire CPU all the time, not
> just on a case-by-case basis.
>
> Any help or advice would be appreciated.

Make sure the correct ide driver is loaded for your chipset.
ide-generic can't use dma and then cpu load will be much higher and disk
access much lower.

You may have to fiddle with the initrd generating scripts if you also
use sata since it would probably only load one driver for the root disk
and let something else take care of the rest later.

This is assuming your chipset is even supported by the kernel you are
running.

Len Sorensen


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to make apps play nice?

Oleg Gritsinevich
In reply to this post by Craig Hagerman
On Sun, Dec 18, 2005 at 11:35:09PM +0900, Craig Hagerman wrote:
[skip]
> $ hdparm /dev/hda
>
> /dev/hda:
>  multcount    = 16 (on)
>  IO_support   =  0 (default 16-bit)
>  unmaskirq    =  0 (off)
>  using_dma    =  0 (off)
^^^^^^^^^^^^^^^^^^^^^^^^^^
[skip]
> (NOTE- sda and sdb are joined up via LVM2 into one big volume)
> So what does all of this mean? Seems that dma is not activated for
> /dev/hda. Does the above mean that is IS active for /dev/sda and
> /dev/sdb?
        Are sda and sdb real SCSI HDDs or SATA? If they are SATA DMA is
turning regarding to their read speeds you showed below.
>
> the -tT flag gives this:
>
> $ hdparm -tT /dev/hda
>
> /dev/hda:
>  Timing cached reads:   2652 MB in  2.00 seconds = 1325.92 MB/sec
>  Timing buffered disk reads:   12 MB in  3.14 seconds =   3.82 MB/sec
                                                            ^^^^^^^^^^^
The performance is very poor (missing of DMA explains it), modern HDD
should give at least 30 MB/sec.

[skip]

> Meanwhile the -d activating option fails:
>
> $ hdparm -d1 /dev/hda
>
> /dev/hda:
>  setting using_dma to 1 (on)
>  HDIO_SET_DMA failed: Operation not permitted
>  using_dma    =  0 (off)
>
>
> Any advice concerning this? How does one get dma activated?
> (Thanks again.)
        It seems that driver you are using for now doesn't support DMA
transfers on your chipset. You have to load kernel module that supports
your IDE chipset.  You can determine your IDE chipset by reading output
of
$ lspci
Then find appropriate module and load it instead of ide_generic.
>
> Craig
>
>

--
With best regards, Oleg Gritsinevich


--
To UNSUBSCRIBE, email to [hidden email]
with a subject of "unsubscribe". Trouble? Contact [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to make apps play nice?

Craig Hagerman
I finally have the conclusion to my problem. (that some apps like find
take up almost all the CPU.) The responses were correct - DMA was not
enabled on my system. I just finished compiling and installing a new
kernel. Now, with DMA enabled, and activated on /dev/hda there is no
longer a problem.

For anyone else interested, my motherboard is an ASUS A8V
lspci reports that the IDE interface is:
IDE interface: VIA Technologies, Inc.
VT82C586A/B/VT82C686/A/B/VT823x/A/C PIPC Bus Master IDE (rev 06)

When compiling a new kernel, the relevant bit is in:

Device Drivers > ATA/ATAPI/MFM/RLL support > VIA82CXX chipset support

Say "yes", recompile etc... After rebooting into the new kernel run:

% hdparm -d1 /dev/hda
Then check it out with
% hdparm /dev/hda
% hdparm -Tt /dev/hda

Cheers,

Craig