sunpermon.docx
- 文档编号:28857907
- 上传时间:2023-07-20
- 格式:DOCX
- 页数:16
- 大小:61.40KB
sunpermon.docx
《sunpermon.docx》由会员分享,可在线阅读,更多相关《sunpermon.docx(16页珍藏版)》请在冰豆网上搜索。
sunpermon
September1995
SystemPerformanceMonitoring
byAdrianCockcroft
WelcomeToThisNewRegularColumn!
I'mreallyexcitedtobestartingaregularcolumnonperformance,tuningandrelatedissues.It'ssomethingIhavewantedtodoforalongtime.Forseveralyears,I'vebeensupportingSun'sSystemEngineerswithperformanceinformation.Iwroteaseriesofwhitepapersonthesubject,andinJanuary1995mybookSunPerformanceandTuning:
SPARCandSolariswaspublishedbySunMicrosystemsPress/PTRPrenticeHall.Whilethebookprovidesacomprehensivereference,itcannotkeepupwiththerapidrateofdevelopmentsustainedatSun.Iintendtousethiscolumntopassonthelatestinformationaboutperformanceandtuning,whileIexplaininmoredetailissuesthatseemtoberepeatedlymisunderstood.Theonlinenatureofthecolumnisagreatasset;therearenoprintingleadtimes;ifImentionatoolorproduct,Icanlinkdirectlytoanotherpage,anftpsite,oravendor;andyoucanusesearchenginestogetinformationinahurry.Itcertainlybeatsrummagingthroughapileofyellowingmagazinesinyourofficelookingforahalf-rememberedarticle,thatcontainsanow-criticalpieceofinformation.
EachmonthIwillfollowthesamegeneralformat.I'llstartwithsomethingbriefandtopical,perhapshighlightingperformance-relatednewsforthemonth.Iwillalsoselectanissueofthemonthtobeexplained,andI'llsummarizewithsomeconciserulesandnon-rules.Anon-ruleissomethingthatmanypeoplethinkshouldbearule,butIthinkyoucanignore.Thisismywayofcancelingoutthefalsefolklore.Aruleisusuallysomethingalongtheselines:
ifyouseethissetofconditions,youhavethisproblem.Thecolumnmightalsoincludeanappropriatetuningtip.Eacharticlewillincludealookathowtousetoolstoinvestigateormanagetheissue.Toolsincludethebundledperformanceutilities,freelyavailabletoolsthatcanbeftp'dovertheInternetand,occasionally,commercialproducts.
Thismonthmysubjectistheprimaryperformancerulesthatcanbeusedtodefinethestateofthemaincomponentsofasystem.I'llalsoshowyouhowtogetafreetoolkitthatautomatestheruleswithaGUIdisplay.NextmonthIintendtocoversomerulesandkerneltweaksthatrequirerootaccess,anddescribeaprogramthatautomatestheprocess,called"virtual_adrian",whichispartofthesametoolkit.Incomingmonths,subjectswillincludeapplicationmonitoringandanalysis;andwhenthewrapscomeoff,myinitialexperienceswithsystemsbasedontheUltraSPARCchip.
SystemPerformanceMonitoring
Abasicreasonwhywemonitortheperformanceofasystemistodefinetheoverallstateofthemachine("It'sOK"or"It'sslow"etc).Thisbreaksdownintothestatesofthesystem'smaincomponentssuchasdisks,nets,RAM,andsoon.Ihaveproducedasetofperformancerulesthatyoucanusetofindoutwhatstateyouarein.ThesearebasedonthosedescribedinAppendixAofmybook,butsincethebookwaspublished,Ihavecodedalltherulesusingaperformancetoolkit.Ihavetestedthemextensivelyandrefinedthemalittle.
Theusualconventionistodefinecolorcodedstatesbasedongreen,amber,red.IhaveextendedthesestatesalittleandthefullsetIuseisasfollows:
∙Whitestate-completelyidle,untouched
∙Bluestate-imbalance,idlewhileotherinstancesoftheresourceareoverloaded
∙Greenstate-noproblem,normaloperatingstate
∙Amberstate-warningcondition
∙Redstate-overloadedorproblemdetected
∙Blackstate-criticalproblemthatmaycauseprocessesorthewholesystemtofail
MonitoringYourSystemwithruletool.se
It'stoomuchworktomanuallyapplyalltherulesmentionedinthisarticle.Luckilyyoudon'thaveto!
IhavecodedeachruleasareusableobjectusingtheSE[SymbolEngine]performancetoolkit(moreontheSEtoolkitnextmonth)andthereisaGUIfrontendtotherules,soyoucanjustrunruletool.seandseethestateofthesystem.Atypicaldisplayisshownhere.Theoverallsystemstateisamber,astwodisksareattheamberwarninglevel,andRAMisgettingshort.Ifyouclickonanypartofthewindow,yougetapopupdisplaythatgivesyoumoredetail.(Don'tbothertryingtoclickontheimagebelow,youneedtoberunningtherealprogram!
).
LongTermMonitoringWithCalendarManager
WhenyouinstalltheSEtoolkityougetanoptiontostart-upsomescriptsautomaticallyatboottime.Oneoftheseiscalledmon_cm.se.Itwakesupevery15minutes,runstherules,andifthereareanyamber,redorblackstates,itcreatesacalendarappointmentfortherootuserindicatingtheproblem.Youcanbrowsethecalendaratanytime,andusingmulti-browse,youcanviewseveralmachinesatonce.Someexampleoutputisshownbelow.Inthiscaseyoucanseethatforthe15minuteperiodfrom5:
59p.m.theoverallstateis"red".Therearenorulesinthe"black"state,theNetsruleisinthe"red"state,andtherearenorulesinthe"amber"state.Youcanalsoseethelistoftimesandstatesforthewholeday.
InthenextexampleIusedmultibrowsetolookatbothanNFSserver(chessie)andaclient(crun)atthesametime.Chessiehasalotofentries,andcrunhasafewthatoverlapandareshownasdarkerbars.Youcanseethepatternthroughtheworkingweek.Thisincludessomeactivityatabout3:
15a.m.thatcorrespondstothenfsfindcommandrunbycronontheNFSserver.
HowtogettheSEToolkit
TheSEToolkitwaswrittenbyRichPettitofSun,Icontributedtherulescode,encouragementandideas.ItrunsonSPARCsystems[workstationsandservers]withSolaris2.3,2.4or2.5beta,andIntelx86systemswithSolaris2.4.ItisfreelyavailablebutitisnotsupportedbySun.
TheRulesinDetail
Thisnextsectiongoesthrougheachruleinturn.Thecomponentsthathavethegreatesteffectonperformancearelistedfirst.Eachrulewasinitiallydefinedintermsoftheoutputofstandardsystemcommands.Forexampletheshorthand"vmstat30.r"meansrunthecommandvmstatwitha30-secondintervalandlookatthecolumnlabeledwithan"r".Ifyouprefer,youcanskipthedetailsandreadaboutmywaitforI/Onon-rule.
TheDiskRule
Therulefordisksisquitecomplex.Thisisbecausetherearemanydisksonasystem,andIchoosetoimplementarulethatcomplainsaboutanimbalanceintheusageofthedisks.IalsopickouttwospecialcasesandrecognizethepresenceofasinglefloppydiskandasingleCDROMdriveasmuchslowerdevices.I'mlookingfordisksthataremorethan20percent(20%)busyandhaveservicetimesofmorethan30-50ms.
Table1DiskRule
RuleforEachDiskDriveLevelAction
------------------------------------------------------------------------------------
(iostat-x30.b<5%)&&(otherdiskswhiteorgreen)WhiteNoProblem
(iostat-x30.b<5%)&&(otherdisksamberorred)Blue1.IdleDisk
(5%<=iostat-x30.b)&&(iostat-x30.svc_t<30ms)GreenNoProblem
(20%<=iostat-x30.b)&&(30ms<=iostat-x30.svc_t<50ms)Amber2.BusyDisk
(20%<=iostat-x30.b)&&(50ms<=iostat-x30.svc_t)Red2.BusyDisk
(20%<=iostat-x30.b)&&(300ms<=iostat-x30.svc_t)&&
(iostat-x30.disk=="fd0"||iostat-x30.disk=="sd6")Amber3.Floppy/CD
(20%<=iostat-x30.b)&&(500ms<=iostat-x30.svc_t)&&
(iostat-x30.disk=="fd0"||iostat-x30.disk=="sd6")Red3.Floppy/CD
0%==iostat-x30.wGreenNoProblem
0% 5%<=iostat-x30.wRed4.SCSIBusy ------------------------------------------------------------------------------------ 1.IdleDisk AnidlediskisawasteofI/Othroughputwhenotherdisksareoverloaded.Rebalancetheloadorstripethisdisktogetherwithabusyone. 2.BusyDisk Abusyorslowdiskreducessystemthroughputandincreasesuserresponsetimes.Rebalancetheloadorstripethisdisktogetherwithanidleone.IftherearenoidledisksavailablethenreducetheI/Oload(perhapsbyincreasingRAMbuffersizes)orconfiguremoredisks. 3.Floppy/CD Topreventthenormalrulesfrombeingappliedtomuchslowerdevices,checkforthedisktypeoffd0,indicatingafloppy.ACDappearsasanormalbutveryslowdiskthathasneverbeenwrittento,andthereisnosimplewaytoexcludeitwithoutaccidentallyexcludingaveryoverloaded,read-onlyharddisk.TheCDisoftenconfiguredastargetnumber6onthefirstSCSIbus,sospecifyingsd6isacrudework-aroundforthisproblem.Ihavefoundthatallowingtentimestheusualservicetimeforthesedevicesworksasareasonablethreshold.Thisisarefinementoftherulegiveninmybook.Iimplementeditinthetoolkitasaseparatemultiplier,defaultingtoten(10)timesslower,foreachtypeofdevice. 4.SCSIBusy Ifdiskcommandshavetowaitbeforetheycanbesenttothedisk,thenthewaitingmaybeduetoanoverloadedSCSIbus.TherearenodirectmeasuresofSCSIbusutilizationlevels.Thiscaseisnotcurrentlyimplementedinthetoolkit. TheNetworkRule Therulefornetworksissimilartothediskrule.Manynetworkinstancesarecombined,andanimbalanceisreported.Aspecialcaseisthattheruleonlydealswithethernets,asitusesethernetcollisionsasametric.IhavenotfoundanysatisfactorymetricsthatindicateoverloadfortokenringorFDDInetworks. Table2NetworkRule RuleforEachNetworkInterfaceLevelAction ---------------------------------------------------------------------------------------------- (0 (100*netstat-i30.output.colls/netstat-i30.output.packets<0.5%) &&(othernetswhiteorgreen) (0
- 配套讲稿:
如PPT文件的首页显示word图标,表示该PPT已包含配套word讲稿。双击word图标可打开word文档。
- 特殊限制:
部分文档作品中含有的国旗、国徽等图片,仅作为作品整体效果示例展示,禁止商用。设计者仅对作品中独创性部分享有著作权。
- 关 键 词:
- sunpermon