免费精品视频一区二区三区学生,被3个黑人老外玩的4p,人妻精品无码中文无码一区无,添女人荫蒂全部过

首頁> 外文學位 >Efficient methods to store and query network data.
【24h】

Efficient methods to store and query network data.

機譯：存儲和查詢網絡數據的有效方法。

獲取原文
獲取原文并翻譯 | 示例
  • 摘要
  • 著錄項
  • 相似文獻
  • 相關主題

摘要

Network data crosses network boundaries in and out and many organizations record traces of network connections for monitoring and investigation purposes. With the increase in network traffic and sophistication of the attacks there is a need for efficient methods to store and query these data. In this dissertation we propose new efficient methods for storing and querying network payload and flow data that can be used to enhance the performance of monitoring and forensic analysis.;We first address the efficiency of various methods used for payload attribution. Given a history of packet transmissions and an excerpt of a possible packet payload, a Payload Attribution System (PAS) makes it feasible to identify the sources, destinations and the times of appearance on a network of all the packets that contained the specified payload excerpt. A PAS, as one of the core components in a network forensics system, enables investigating cybercrimes on the Internet, by, for example, tracing the spread of worms and viruses, identifying who has received a phishing email in an enterprise, or discovering which insider allowed an unauthorized disclosure of sensitive information. Considering the increasing volume of network traffic in today's networks it is infeasible to effectively store and query all the actual packets for extended periods of time for investigations. In this dissertation we focus on extremely compressed digests of payload data, we analyze the existing approaches and propose several new methods for payload attribution which utilize Rabin fingerprinting, shingling, and winnowing. Our best methods allow building payload attribution systems which provide data reduction ratios greater than 100:1 while supporting efficient queries with very low false positive rates. We demonstrate the properties of the proposed methods and specifically analyze their performance and practicality when used as modules of a network forensics system.;Consequently, we propose a column oriented storage infrastructure for storing historical network flow data. Transactional row-oriented databases provide satisfactory query performance for network flow data collected only over a period of several hours. In many cases, such as the detection of sophisticated coordinated attacks, it is crucial to query days, weeks or even months worth of disk resident historical data rapidly. For such monitoring and forensics queries, row oriented databases become I/O bound due to long disk access times. Furthermore, their data insertion rate is proportional to the number of indexes used, and query processing time is increased when it is necessary to load unused attributes along with the used ones. To overcome these problems in this dissertation we propose a new column oriented storage infrastructure for network flow records and present the performance evaluation of a prototype storage system implementation called NetStore. The system is aware of network data semantics and access patterns, and benefits from the simple column oriented layout without the need to meet general purpose databases requirements. We show that NetStore can potentially achieve more than ten times query speedup and ninety times less storage requirements compared to traditional row-stores, while it performs better than existing open source column-stores for network flow data.;Finally, we propose an efficient querying framework to represent, implement and execute forensics and monitoring queries faster on historical network flow data. Using efficient filtering methods, the query processing algorithms can improve the query runtime performance up to an order of magnitude for simple filtering and aggregation queries, and up to six times for batch complex queries when compared to naive approaches. Additionally, we propose a simple SQL extension that implements a subset of standard SQL commands and operators and a small set of features useful for network monitoring and forensics. The presented query processing engine together with a column storage infrastructure create a complete system for storing and querying network flow data efficiently when used for monitoring and forensic analysis.
機譯：網絡數據跨入和跨出網絡邊界，許多組織記錄網絡連接的痕跡以進行監視和調查。隨著網絡流量的增加和攻擊的復雜性，需要一種有效的方法來存儲和查詢這些數據。本文提出了一種新的有效方法來存儲和查詢網絡有效載荷和流量數據，可用于提高監視和取證分析的性能。給定數據包傳輸的歷史記錄和可能的數據包有效內容摘錄，有效負載歸因系統（PAS）使得識別包含指定有效負載摘錄的所有數據包的源，目的地和出現在網絡上的時間變得可行。作為網絡取證系統的核心組件之一，PAS可以例如通過跟蹤蠕蟲和病毒的傳播，識別誰在企業中收到網絡釣魚電子郵件或發現哪個內部人員來調查Internet上的網絡犯罪。允許未經授權披露敏感信息。考慮到當今網絡中網絡流量的不斷增長，有效地存儲和查詢所有實際數據包較長時間以進行調查是不可行的。本文重點研究了有效載荷數據的極壓縮摘要，分析了現有方法，提出了幾種利用拉賓指紋，帶狀疊層和風選的有效載荷歸因方法。我們的最佳方法允許構建有效載荷歸因系統，該系統提供大于100：1的數據縮減率，同時以極低的誤報率支持有效的查詢。我們演示了所提出方法的特性，并特別分析了它們在用作網絡取證系統的模塊時的性能和實用性。因此，我們提出了一種用于存儲歷史網絡流數據的面向列的存儲基礎架構。面向行的事務型數據庫僅在幾個小時的時間內對收集的網絡流數據提供了令人滿意的查詢性能。在許多情況下，例如檢測復雜的協同攻擊，快速查詢幾天，幾周甚至幾個月的磁盤駐留歷史數據至關重要。對于此類監視和取證查詢，由于磁盤訪問時間長，面向行的數據庫已成為I / O綁定。此外，它們的數據插入率與所使用的索引數成正比，并且當有必要將未使用的屬性與已使用的屬性一起加載時，查詢處理時間會增加。為了克服這些問題，我們提出了一種面向列的，用于網絡流記錄的存儲基礎架構，并提出了一種稱為NetStore的原型存儲系統實現的性能評估。該系統知道網絡數據的語義和訪問模式，并且受益于簡單的面向列的布局，而無需滿足通用數據庫的要求。我們證明，與傳統的行存儲相比，NetStore可以潛在地實現超過10倍的查詢速度提升和90倍的存儲需求降低，同時它比現有的開放源列存儲對網絡流數據的性能要好。最后，我們提出了一種有效的查詢表示，實施和執行取證和監視歷史網絡流量數據的查詢更快的框架。與簡單的方法相比，使用高效的過濾方法，查詢處理算法可以將查詢運行時性能提高多達一個數量級（對于簡單過濾和聚合查詢），對于批處理復雜查詢，則可以提高六倍。此外，我們提出了一個簡單的SQL擴展，它實現了標準SQL命令和運算符的子集，以及一小部分對網絡監視和取證有用的功能。提出的查詢處理引擎與列存儲基礎結構一起創建了一個完整的系統，用于在用于監視和取證分析時有效地存儲和查詢網絡流數據。

著錄項

  • 作者

    Giura, Paul.;

  • 作者單位

    Polytechnic Institute of New York University.;

  • 授予單位 Polytechnic Institute of New York University.;
  • 學科 Computer Science.
  • 學位 Ph.D.
  • 年度 2010
  • 頁碼 141 p.
  • 總頁數 141
  • 原文格式 PDF
  • 正文語種 eng
  • 中圖分類
  • 關鍵詞

相似文獻

  • 外文文獻
  • 中文文獻
  • 專利
獲取原文

客服郵箱：kefu@zhangqiaokeyan.com

京公網安備：11010802029741號 ICP備案號：京ICP備15016152號-6 六維聯合信息科技 (北京) 有限公司?版權所有

  • 客服微信

  • 服務號

主站蜘蛛池模板： 津市市| 平度市| 北碚区| 泗洪县| 玛沁县| 兴宁市| 漳州市| 和硕县| 密云县| 南靖县| 金坛市| 遂溪县| 元氏县| 莆田市| 工布江达县| 崇礼县| 孟津县| 凤阳县| 鹿泉市| 泰顺县| 钟山县| 轮台县| 喀喇沁旗| 五莲县| 蓝田县| 怀仁县| 乡城县| 玉龙| 上杭县| 色达县| 沈丘县| 梅州市| 宜兰市| 西乌珠穆沁旗| 深水埗区| 渑池县| 泸州市| 惠东县| 确山县| 安吉县| 天津市|