亚洲国产日韩欧美一区二区三区,精品亚洲国产成人av在线,国产99视频精品免视看7,99国产精品久久久久久久成人热,欧美日韩亚洲国产综合乱

The correct answer is this: sort|uniq -c

a.b.c.d

Deal with a first.

All files are divided into multiple reads (such as 255 times).

The records are then stored in 255 files, corresponding to a (such as 1.b.c.d; 2.b.c.d;)

This way you can get the largest ten.

Next, deal with b.

. . .

Finally got the answer.

This kind of problem can use max-min heap.

Basic idea

1. 對(duì)所有的ip進(jìn)行分組，統(tǒng)計(jì)出現(xiàn)次數(shù)，ip做key，出現(xiàn)次數(shù)做val，保存在關(guān)聯(lián)數(shù)組中

2. 數(shù)組按照鍵值（出現(xiàn)次數(shù)）降序排序，取出前 10 條記錄

Although it’s not java語(yǔ)言描述的，但javascript描述, I should be able to understand it. Anyway, I have tested it and it can achieve your effect:

The following is the complete example code:

    var data = [123.22.22.187 , 123.22.22.187 ....];  // 等價(jià)于日志中每行ip
    var data = getCount(data); // 對(duì)所有的ip進(jìn)行分組，統(tǒng)計(jì)出現(xiàn)次數(shù)
                               // ip做key，出現(xiàn)次數(shù)做val，保存在關(guān)聯(lián)數(shù)組中
    var rel  = getRecord(data , 10 , 'desc'); // 這個(gè)就是你想的結(jié)果了
    
    /*
     * ip 分組并統(tǒng)計(jì)次數(shù)的具體實(shí)現(xiàn)
     * @param Array data  日志文件中存放的那些所有ip組成的數(shù)組
     * @return Object
     */
    function getCount(data){
        var rel= {}; // 根據(jù)ip進(jìn)行分組后的結(jié)果，由于js沒(méi)有關(guān)聯(lián)數(shù)組
                      // 所以用對(duì)象替換
        var i;
        var count;
        var n;
        var cur;
        var curC;
        // 判斷是否已經(jīng)統(tǒng)計(jì)過(guò)了的
        var checkIsCount = function(ip){
            var k;
            
            for (k in rel)
                {
                    if (k === ip) {
                        return true;
                    }
                }
            
            return false;
        };
        
        // 每次固定一個(gè) ip
        // 先判斷當(dāng)前 ip 在 rel 中是否已經(jīng)存在（排除已被統(tǒng)計(jì)的ip，避免重復(fù)統(tǒng)計(jì)）
        // 不存在的時(shí)候，開(kāi)始統(tǒng)計(jì)其出現(xiàn)次數(shù)
        for (i = 0; i < data.length; ++i)
            {
                cur = data[i];
                count = 1;
                
                if (!checkIsCount(cur)) {
                    for (n = i + 1; n < data.length; ++n)
                        {
                            curC = data[n];
    
                            if (cur === curC) {
                                count++;
                            }    
                        }
    
                     rel[cur] = count;      
                }
            }
            
       // 保存了所有不同ip，及其出現(xiàn)次數(shù)的對(duì)象
       console.log('ip進(jìn)行了分組并統(tǒng)計(jì)后的結(jié)果：' , rel);
       return rel;
    }
    
    /*
     * 數(shù)組按照鍵值（出現(xiàn)次數(shù)）降序排序，取出前 len 條記錄
     * @param Object data 分組并統(tǒng)計(jì)出現(xiàn)次數(shù)后的對(duì)象
     * @param Number len  取出多少條
     * @param String sortType 升序還是降序(asc | desc)
     * @return Array
     */
    function getRecord(data , len , sortType){
        var rel = {};
        var result = [];
        var k;
        var k1;
        var curKey;
        var count = 0;
        // 檢查 ip 是否已被獲取
        var checkSorted = function(ip){
            var k;
            
            for (k in rel)
                {
                    if (k === ip) {
                        return true;
                    }
                }
                
            return false;
        };
    
        // 排序算法：選擇排序
        for (k in data)
            {
                
               if (count >= len) {
                   break;
               }
               
               curKey = k;
               
               if (!checkSorted(k)) {
                   for (k1 in data) 
                       {
                           if (!checkSorted(k1)) {
                              if (sortType === 'asc' ? data[curKey] > data[k1] : data[curKey] < data[k1]) {
                                  curKey = k1;
                              } 
                           }
                       }
                   
                   
                   // 按照鍵值排序后的最大值|最小值
                   rel[curKey] = data[curKey];
    
                   // 返回可視化結(jié)果
                   result.push(curKey);
                   
                   // 獲取的長(zhǎng)度增加
                   count++
               }
            }
    
        // 按照鍵值排序后的 len 長(zhǎng)度的記錄
        console.log('取出指定長(zhǎng)度記錄的實(shí)際結(jié)果：' , rel);
        console.log('取出指定長(zhǎng)度記錄的可視化結(jié)果：' , result);
        return result;
    }

map-reduce or sort | uniq -c
Obviously use hadoop (map-reduce)?

Sort it first, it will be easier to deal with after sorting.

This question should be broken into two parts to understand,
1. Reading a large file at once is inefficient
2. Find the top 10 records with the most occurrences
Question 1: For reading large files, you can use java AIO Implementation method reads files asynchronously
Question 2: LindedList needs to be extended. Each time a record is appended, first check whether a record exists, and if so, push the record position forward (this is more similar to the java memcached caching algorithm)

These types of problems all have the same idea, "divide and conquer".
Each IP is equivalent to a 32-bit integer number. Traverse all the IPs in the log, divide them into n files according to the value range and save them (make sure each file is small enough). In this case, the same IP will definitely be assigned to the same file. In a file; then process these n files one by one, count the top 10 IPs that appear in each file; finally get the top 10 from these 10*n IPs.

Use regular expressions to determine the first number at the beginning of the IP:

1.2.3\n
4.5.6\n
1.2.3\n

The number at the beginning of the line that appears the most is the most frequent IP