亚洲一区欧美在线,日韩欧美视频免费观看,色戒的三场床戏分别是在几段,欧美日韩国产在线人成

基于Solr的農(nóng)田數(shù)據(jù)索引方法與大數(shù)據(jù)平臺構(gòu)建
作者:
作者單位:

作者簡介:

通訊作者:

中圖分類號:

基金項目:

國家重點(diǎn)研發(fā)計劃項目(2017YFD0700601)


Index Method of Farmland Data Based on Solr and Construction of Big Data Platform
Author:
Affiliation:

Fund Project:

  • 摘要
  • |
  • 圖/表
  • |
  • 訪問統(tǒng)計
  • |
  • 參考文獻(xiàn)
  • |
  • 相似文獻(xiàn)
  • |
  • 引證文獻(xiàn)
  • |
  • 資源附件
  • |
  • 文章評論
    摘要:

    針對農(nóng)田數(shù)據(jù)在高吞吐量、高并發(fā)、多條件處理過程中易產(chǎn)生運(yùn)算負(fù)載大、響應(yīng)速度慢等難題,研究了負(fù)載均衡大規(guī)模集群數(shù)據(jù)處理技術(shù),優(yōu)化了多條件檢索時Hbase農(nóng)田數(shù)據(jù)庫,提出了基于Solr的二級非主鍵索引方法,搭建了基于Hadoop的農(nóng)田大數(shù)據(jù)平臺,采用農(nóng)機(jī)深松、植保、保護(hù)性耕作等8種作業(yè)生成的100TB數(shù)據(jù)對平臺進(jìn)行了檢索實(shí)驗(yàn)和壓力測試實(shí)驗(yàn)。實(shí)驗(yàn)結(jié)果表明,多條件檢索時,優(yōu)化后的技術(shù)模型在數(shù)據(jù)規(guī)模達(dá)到5×107條時,系統(tǒng)的響應(yīng)時間小于1s,優(yōu)化的性能與原生Hbase相比提高了3倍;在模擬用戶達(dá)到5×105次時,系統(tǒng)的QPS及TPS提高了1倍左右、RT提高了2.5倍,系統(tǒng)的平均響應(yīng)時間為183ms。本研究解決了高吞吐量、高并發(fā)導(dǎo)致農(nóng)田數(shù)據(jù)檢索效率低的問題,提高了海量農(nóng)田數(shù)據(jù)實(shí)時處理的計算能力。

    Abstract:

    Aiming at the problems of high throughput, high concurrency and slow response of farmland data in the process of multi-condition processing, such as high computational load and slow response speed. The data processing technology of load balancing large-scale cluster was studied, the Hbase farmland database in multi-condition retrieval was optimized, a two-level non-primary key index method was put forward based on Solr, and a large farmland data platform was buildt based on Hadoop. The 100TB data was generated by eight operations, such as subsoiling, plant protection and conservation tillage, and those data was retrieved and tested on the platform based on Hadoop. The experimental results showed that the response time of the optimized technical model was less than 1 s when the concurrent volume of farm data was 50 million, and the performance of the optimized model was improved by about four times compared with the original Hbase. When the concurrent volume of simulated users were 500000, the query per second (QPS)and transaction per second (TPS)of the system were increased by about one time, the response time (RT) of the system was increased by 2.5 times, and the average response time was 183ms. To a certain extent, this system solved the problem of low efficiency of farmland data retrieval caused by high throughput and concurrency, and improved the computing ability of real-time processing of massive farmland data.

    參考文獻(xiàn)
    相似文獻(xiàn)
    引證文獻(xiàn)
引用本文

苑嚴(yán)偉,冀福華,趙博,姜含露,王猛,樊學(xué)謙.基于Solr的農(nóng)田數(shù)據(jù)索引方法與大數(shù)據(jù)平臺構(gòu)建[J].農(nóng)業(yè)機(jī)械學(xué)報,2019,50(11):186-192. YUAN Yanwei, JI Fuhua, ZHAO Bo, JIANG Hanlu, WANG Meng, FAN Xueqian. Index Method of Farmland Data Based on Solr and Construction of Big Data Platform[J]. Transactions of the Chinese Society for Agricultural Machinery,2019,50(11):186-192.

復(fù)制
分享
文章指標(biāo)
  • 點(diǎn)擊次數(shù):
  • 下載次數(shù):
  • HTML閱讀次數(shù):
  • 引用次數(shù):
歷史
  • 收稿日期:2019-05-07
  • 最后修改日期:
  • 錄用日期:
  • 在線發(fā)布日期: 2019-11-10
  • 出版日期: