linux下的fastcopy
在拷贝数据的时候,如果遇到一堆小文件,即使总数据量很小,拷贝也会很费时,在win下,有一个软件叫fastcopy,可以实现快速拷贝,据说使用了一种叫内存映射的技术,那么在Linux下有没有呢?
我找了半天,原来LInux下根本不需要单独的软件,用现有的命令组合就OK了!
有高手在两个服务器间传数据,使用命令,实现了快速传送数据。
http://www.4bcj.com/post/2008/01/Fast-File-Copy—Linux!.aspx
All good ideas come out of necessity. We were cloning an instance of
Oracle Financials from one server to another. There are a LOT of files
under < 1k and the copy takes forever. Yesterday the copy was kicked
off using SCP - there was 39GBs to copy over a gigabit switch. This
should have been less than 10 minutes but actually took over 8 hours
because of all the small files. The copy failed and we needed to fix
the problem and copy it a lot faster (30 minutes) today.
After clearing up 10 GBs of log files, we were left with hundreds of
thousands of small files that were going to slow us down. We couldn’t
tarball the file because of a lack of space on the source server. I
started searching around and found this nifty tip that takes our
encryption and streams all the files as one large file:
This requires netcat on both servers.
Destination box: nc -l -p 2342 | tar -C /target/dir -xzf -
Source box: tar -cz /source/dir | nc Target_Box 2342
It’s been about 4 minutes and I’m already 1/3 of the way done!
感谢本校LInux版主ashmer,写了这个shell脚本,可以实现fastcopy, 与诸位共享.
#!/bin/sh
# file: fastcp.sh
source="$1"
target="$2"
if [ "x$1" == "x" ] || [ "x$2" == "x" ] ; then
echo "$0 SOURCE_DIR TARGET_DIR"
exit 1
fi
exec tar -cp "$source" -f - | tar -xpv -C "$target" -f -
#end
PS: 测试了一番,速度竟然由cp的0m21.270s降低至0m6.984s,真是够快的。。。
相关文档:
一:前言
最近在研究android的sensor driver,主要是E-compass,其中用到了Linux input子系统.在网上也看了很多这方面的资料,感觉还是这篇分析的比较细致透彻,因此转载一下以便自己学习,同时和大家分享!
(这篇博客主要是以键盘驱动为例的,不过讲解的是Linux Input Subsystem,可以仔细的研究一下!)
键盘驱动将检 ......
在linux下如何把一个lvm的文件系统/home减到1G。比aix下麻烦,aix下直接把文件系统缩到某个值,对应的lv自动就减少了,不像linux要先缩小文件系统,再缩小lv。
suse1:~ # df -m
Filesystem 1M-blocks Used Available Use% Mounted on
/dev/mapper/system-root
5040 299 ......
Linux LVM 的使用详解
http://blog.csdn.net/icejadelove/archive/2009/10/30/4747657.aspx
摘要:Linux用户安装Linux操作系统时遇到的一个最常见的难以决定的问题就是如何正确地给评估各分区大小,以分配合适的硬盘空间。而遇到出现某个分区空间耗尽时,解决的方法通常是使用符号链接,或者使用调整分区大小的工具(比如P ......
Linux 引导过程内幕
引导 Linux® 系统的过程包括很多阶段。不管您是引导一个标准的 x86 桌面系统,还是引导一台嵌入式的 PowerPC® 机器,很多流程都惊人地相似。本文将探索 Linux 的引导过程,从最初的引导到启动第一个用户空间应用程序。在本文介绍的过程中,您将学习到各种与引导有关的主题, ......