大数据工具篇之Hive与HBase整合完整教程Word文档格式.docx
- 文档编号:16187696
- 上传时间:2022-11-21
- 格式:DOCX
- 页数:61
- 大小:52.66KB
大数据工具篇之Hive与HBase整合完整教程Word文档格式.docx
《大数据工具篇之Hive与HBase整合完整教程Word文档格式.docx》由会员分享,可在线阅读,更多相关《大数据工具篇之Hive与HBase整合完整教程Word文档格式.docx(61页珍藏版)》请在冰豆网上搜索。
!
--
4LicensedtotheApacheSoftwareFoundation(ASF)underoneormore
5contributorlicenseagreements.SeetheNOTICEfiledistributedwith
6thisworkforadditionalinformationregardingcopyrightownership.
7TheASFlicensesthisfiletoYouundertheApacheLicense,Version2.0
8(the"
License"
);
youmaynotusethisfileexceptincompliancewith
9theLicense.YoumayobtainacopyoftheLicenseat
10
11http:
//www.apache.org/licenses/LICENSE-2.0
12
13Unlessrequiredbyapplicablelaworagreedtoinwriting,software
14distributedundertheLicenseisdistributedonan"
ASIS"
BASIS,
15WITHOUTWARRANTIESORCONDITIONSOFANYKIND,eitherexpressorimplied.
16SeetheLicenseforthespecificlanguagegoverningpermissionsand
17limitationsundertheLicense.
18-->
19
20<
configuration>
21
22<
--WARNING!
ThisfileisprovidedfordocumentationpurposesONLY!
-->
23<
AnychangesyoumaketothisfilewillbeignoredbyHive.-->
24<
Youmustmakeyourchangesinhive-site.xmlinstead.-->
25
26
27<
--HiveExecutionParameters-->
28<
property>
29<
name>
mapred.reduce.tasks<
/name>
30<
value>
-1<
/value>
31<
Thedefaultnumberofreducetasksperjob.Typicallyset
32toaprimeclosetothenumberofavailablehosts.Ignoredwhen
33mapred.job.trackeris"
local"
.Hadoopsetthisto1bydefault,whereashiveuses-1asitsdefaultvalue.
34Bysettingthispropertyto-1,Hivewillautomaticallyfigureoutwhatshouldbethenumberofreducers.
35<
36<
/property>
37
38<
39<
hive.exec.reducers.bytes.per.reducer<
40<
1000000000<
41<
sizeperreducer.Thedefaultis1G,i.eiftheinputsizeis10G,itwilluse10reducers.<
42<
43
44<
45<
hive.exec.reducers.max<
46<
999<
47<
maxnumberofreducerswillbeused.Iftheone
48specifiedintheconfigurationparametermapred.reduce.tasksis
49negative,hivewillusethisoneasthemaxnumberofreducerswhen
50automaticallydeterminenumberofreducers.<
51<
52
53<
54<
hive.cli.print.header<
55<
false<
56<
Whethertoprintthenamesofthecolumnsinqueryoutput.<
57<
58
59<
60<
hive.cli.print.current.db<
61<
62<
Whethertoincludethecurrentdatabaseinthehiveprompt.<
63<
64
65<
66<
hive.cli.prompt<
67<
hive<
68<
Commandlinepromptconfigurationvalue.Otherhiveconfcanbeusedin
69thisconfigurationvalue.Variablesubstitutionwillonlybeinvokedatthehive
70clistartup.<
71<
72
73<
74<
hive.exec.scratchdir<
75<
/tmp/hive-${user.name}<
76<
ScratchspaceforHivejobs<
77<
78
79<
80<
hive.exec.local.scratchdir<
81<
/tmp/${user.name}<
82<
LocalscratchspaceforHivejobs<
83<
84
85<
86<
hive.test.mode<
87<
88<
whetherhiveisrunningintestmode.Ifyes,itturnsonsamplingandprefixestheoutputtablename<
89<
90
91<
92<
hive.test.mode.prefix<
93<
test_<
94<
ifhiveisrunningintestmode,prefixestheoutputtablebythisstring<
95<
96
97<
--Iftheinputtableisnotbucketed,thedenominatorofthetablesampleisdeterminiedbytheparameterbelow-->
98<
--Forexample,thefollowingquery:
99<
--INSERTOVERWRITETABLEdest-->
100<
--SELECTcol1fromsrc-->
101<
--wouldbeconvertedto-->
102<
--INSERTOVERWRITETABLEtest_dest-->
103<
--SELECTcol1fromsrcTABLESAMPLE(BUCKET1outof32onrand
(1))-->
104<
105<
hive.test.mode.samplefreq<
106<
32<
107<
ifhiveisrunningintestmodeandtableisnotbucketed,samplingfrequency<
108<
109
110<
111<
hive.test.mode.nosamplelist<
112<
113<
ifhiveisrunningintestmode,dontsampletheabovecommaseperatedlistoftables<
114<
115
116<
117<
hive.metastore.uris<
118<
119<
Thrifturifortheremotemetastore.Usedbymetastoreclienttoconnecttoremotemetastore.<
120<
121
122<
123<
javax.jdo.option.ConnectionURL<
124<
jdbc:
derby:
;
databaseName=metastore_db;
create=true<
125<
JDBCconnectstringforaJDBCmetastore<
126<
127
128<
129<
javax.jdo.option.ConnectionDriverName<
130<
org.apache.derby.jdbc.EmbeddedDriver<
131<
DriverclassnameforaJDBCmetastore<
132<
133
134<
135<
javax.jdo.PersistenceManagerFactoryClass<
136<
org.datanucleus.jdo.JDOPersistenceManagerFactory<
137<
classimplementingthejdopersistence<
138<
139
140<
141<
javax.jdo.option.DetachAllOnCommit<
142<
true<
143<
detachesallobjectsfromsessionsothattheycanbeusedaftertransactioniscommitted<
144<
145
146<
147<
javax.jdo.option.NonTransactionalRead<
148<
149<
readsoutsideoftransactions<
150<
151
152<
153<
javax.jdo.option.ConnectionUserName<
154<
APP<
155<
usernametouseagainstmetastoredatabase<
156<
157
158<
159<
javax.jdo.option.ConnectionPassword<
160<
mine<
161<
passwordtouseagainstmetastoredatabase<
162<
163
164<
165<
javax.jdo.option.Multithreaded<
166<
167<
SetthistotrueifmultiplethreadsaccessmetastorethroughJDOconcurrently.<
168<
169
170<
171<
datanucleus.connectionPoolingType<
172<
DBCP<
173<
UsesaDBCPconnectionpoolforJDBCmetastore<
174<
175
176<
177<
datanucleus.validateTables<
178<
179<
validatesexistingschemaagainstcode.turnthisonifyouwanttoverifyexistingschema<
180<
181
182<
183<
datanucleus.validateColumns<
184<
185<
186<
187
188<
189<
datanucleus.validateConstraints<
190<
191<
192<
193
194<
195<
datanucleus.storeManagerType<
196<
rdbms<
197<
metadatastoretype<
198<
199
200<
201<
datanucleus.autoCreateSchema<
202<
203<
createsnecessaryschemaonastartupifonedoesn'
texist.setthistofalse,aftercreatingitonce<
204<
205
206<
207<
datanucleus.autoStartMechanismMode<
208<
checked<
209<
throwexceptionifmetadatatablesareincorrect<
210<
211
212<
213<
datanucleus.transactionIsolation<
214<
read-committed<
215<
Defaulttransactionisolationlevelforidentitygeneration.<
216<
217
218<
219<
datanucleus.cache.level2<
220<
221<
Usealevel2cache.Turnthisoffifmetadataischangedindependentlyofhivemetastoreserver<
222<
223
224<
225<
datanucleus.cache.level2.type<
226<
SOFT<
227<
SOFT=softreferencebasedcache,WEAK=weakreferencebasedcache.<
228<
229
230<
231<
datanucleus.identifierFactory<
232<
datanucleus<
233<
Nameoftheidentifierfactorytousewhengeneratingtable/columnnamesetc.'
datanucleus'
isusedforbackwardcompatibility<
234<
235
236<
237<
datanucleus.plugin.pluginRegistryBundleCheck<
238<
LOG<
239<
Defineswhathappenswhenpluginbundlesarefoundandareduplicated[EXCEPTION|LOG|NONE]<
240<
241
242<
243<
hive.metastore.warehouse.dir<
244<
/user/hive/warehouse<
245<
locationo
- 配套讲稿:
如PPT文件的首页显示word图标,表示该PPT已包含配套word讲稿。双击word图标可打开word文档。
- 特殊限制:
部分文档作品中含有的国旗、国徽等图片,仅作为作品整体效果示例展示,禁止商用。设计者仅对作品中独创性部分享有著作权。
- 关 键 词:
- 数据 工具 Hive HBase 整合 完整 教程