kindle manager

CLIP.md

@@ -0,0 +1,6 @@
+TYPE|BOOKNAME|AUTHOR|MARKTIME|CONTENT
+--|--|--|--|--
+HL|薛兆丰经济学讲义|薛兆丰|2020/1/13 8:11:05|么到底什么叫边际？边际就是“新增”带来的“新增”。 例如，边际成本就是每新增一个单位产品所需要付出的新增成本；边际收入是每多卖一个产品能够带来的新增收入；边际产量是每新增一份投入所带来的新增产量；边际效用是每消耗一个单位的商品所能带来的新增享受。
+HL|薛兆丰经济学讲义|薛兆丰|2020/1/30 10:23:58|一个国家很大，贫富有差距，并非每个学校和家长都能负担得起这样标准的校车。标准太高，就会逼着很多学校，尤其是农村的学校放弃提供校车，家长们就只能使用安全性能更低的交通工具，比如自己骑自行车或雇用黑车等，结果是孩子们享受到的安全保障反而降低了。--CG注:山寨 假货 问题
+HL|庆余年(精校版）|猫腻|2020/1/19 20:00:29|园子里的护卫能掺多少人就掺多少人，我会派人盯着
+HL|庆余年(精校版）|猫腻|2020/1/20 19:57:10|叶灵儿叹了口

kman.py

@@ -823,6 +823,9 @@ class Util:
 
 
 if __name__=='__main__':
+
+    """
+    # XXX move to unitest.kman.py
     #books = defaultdict(dict)
     km = kMan()
     books = km.import_clips()
@@ -853,3 +856,6 @@ if __name__=='__main__':
 
     # print data with json format
     logger.debug(json.dumps(books, indent=4, sort_keys=True, ensure_ascii=False))
+    """
+
+    pass

kmanapp.py

@@ -867,35 +867,6 @@ class nTableModel(QAbstractTableModel):
 
 if __name__ == "__main__":
 
-    """
-    frozen = 'not'
-    if getattr(sys, 'frozen', False):
-        # we are running in a bundle
-        frozen = 'ever so'
-        bundle_dir = sys._MEIPASS
-    else:
-        # we are running in a normal Python environment
-        bundle_dir = os.path.dirname(os.path.abspath(__file__))
-    print( 'we are',frozen,'frozen')
-    print( 'bundle dir is', bundle_dir )
-    print( 'sys.argv[0] is', sys.argv[0] )
-    print( 'sys.executable is', sys.executable )
-    print( 'os.getcwd is', os.getcwd() )
-
-    print('sys.path[0]', sys.path[0])
-    print('sys.argv[0]', sys.argv[0])
-    print('os.path.realpath(sys.executable)', os.path.realpath(sys.executable))
-    print('os.path.realpath(sys.argv[0]))', os.path.realpath(sys.argv[0]))
-    print('os.path.dirname(os.path.realpath(sys.executable))',
-            os.path.dirname(os.path.realpath(sys.executable)))
-    print('os.path.dirname(os.path.realpath(sys.argv[0]))',
-            os.path.dirname(os.path.realpath(sys.argv[0])))
-    print('os.path.dirname(os.path.abspath(__file__))',
-            os.path.dirname(os.path.abspath(__file__)))
-    util = Util()
-    print('get_app_path',util.get_app_path())
-    """
-
     app = QApplication(sys.argv)
     kmw = kmanWindow()
     icon = QIcon()
@@ -921,7 +892,7 @@ if __name__ == "__main__":
     """
 
     # loop check kindle is connected or not
-    # BUG  to be implement XXXX
+    # BUG, to be implement XXXX
     """
     try:
         t = threading.Thread(target=kmw.check_kindle_status)

makepkg.md

@@ -1,7 +1,6 @@
 
 # 打包问题
 
-
 1. 'ValueError: too many values to unpack (expected 2)
 A: [资源文件打包配置](https://blog.csdn.net/weixin_42052836/article/details/82315118)
 资源文件包括打包的python项目使用的相关文件，如图标文件，文本文件等。对于此类资源文件的打包需要设置Analysis的datas，如例子所示datas接收元组：datas=[(SETUP_DIR+'lib\\icon','lib\\icon'),(SETUP_DIR+'data','data')]。元组的组成为(原项目中资源文件路径，打包后路径)，例子中的(SETUP_DIR+'lib\\icon','lib\\icon')表示从D:\\install_test\\FASTPLOT\\lib\\icon下的图标文件打包后放入打包结果路径下的lib\\icon目录。
@@ -12,22 +11,35 @@ FAIL - LSOpenURLsWithRole() failed with error -10810 for the file /Users/mark/pe
 OK
 A:
     1. 到其他路径区执行(kmanenv) [gavin@gavin-2 penv]$py kman/kmanapp.py看有没有问题
-    2. 所有路径合并，不用+, 用os.path.join()
+    1. 所有路径合并，不用+, 用os.path.join()
-    3. 
+    1.
-
+    ```
+    frozen = 'not'
+    #CURRPATH = os.path.dirname(os.path.realpath(sys.argv[0]))
+    CURRPATH = ''
+    if getattr(sys, 'frozen', False):
+        # we are running in a bundle
+        frozen = 'ever so'
+        CURRPATH = sys._MEIPASS
+        os.chdir(CURRPATH)
+    else:
+        # we are running in a normal Python environment
+        CURRPATH = os.path.dirname(os.path.abspath(__file__))
+    ```
 1.  (kmanenv) [gavin@gavin-2 tkman]$pyinstaller --distpath Release -w -c -i kmanapp.ico kmanapp.spec
 21750 ERROR: Can not find path ./libshiboken2.abi3.5.15.dylib (needed by /Users/mark/.virtualenvs/kmanenv/lib/python3.7/site-packages/PySide2/QtGui.abi3.so)
 A: rm -fr __pycache__, 重新打包
 
-
 1.  (kmanenv) [gavin@gavin-2 kman]$Release/kmanapp.app/Contents/MacOS/kmanapp
 NameError: name 'kMan' is not defined
 A:   pathex=['/Users/mark/penv/kman',]
      Analysis(['kmanapp.py'], ==>
      Analysis(['kman.py', 'kmanapp.py', 'kmanapp_rc.py', 'mainwindow.py', 'mtable.py', 'parseweb.py', ],
-
+     ...
+     **这里有个问题，打开app是会运行所有('kman.py', 'kmanapp.py', 'kmanapp_rc.py', 'mainwindow.py', 'mtable.py', 'parseweb.py',)py文件里的main，所以在打包时要把不是入口py程序的main删除或注释掉**
 
 # reference
 1. [ModuleNotFoundError: No module named 'pkg_resources.py2_warn' ](https://blog.csdn.net/qq_40608730/article/details/104864943)
 https://www.cnblogs.com/yemeng/p/6253097.html
 https://blog.csdn.net/SCDS_Zyx/article/details/82052396
+

mtable.py

@@ -80,6 +80,8 @@ class mTable():
 
 if __name__=='__main__':
 
+    """
+    # move to unitest.kman.py
     print('\n---------------------')
     data = [['Ohiox','Ohio','Ohio','Nevada','Nevada'],
             [2000,2001,2002,2001,2002],
@@ -126,3 +128,5 @@ if __name__=='__main__':
     print('== frame.shape[1]\n', mt.get_num_columns())
     print('== frame.columns\n', mt.get_columns())
     print('== frame.index\n', mt.get_index())
+    """
+    pass

parseweb.py

@@ -327,6 +327,8 @@ class bookInfoSpide():
 
 if __name__ == '__main__':
 
+    """
+    # XXX move to unitest.kman.py
     spide = bookInfoSpide()
 
     for bkname in testbooks:
@@ -339,4 +341,6 @@ if __name__ == '__main__':
         logger.debug('================ {} ================'.format(bkname))
         logger.debug(json.dumps(bkinfo,indent=2, ensure_ascii=False))
         logger.debug(json.dumps(filter_bkinfo,indent=2, ensure_ascii=False))
+    """
 
+    pass

searchtitle.md

@@ -0,0 +1,5 @@
+TYPE|BOOKNAME|AUTHOR|MARKTIME|CONTENT
+--|--|--|--|--
+HL|薛兆丰经济学讲义|薛兆丰|2020/1/13 8:11:05|么到底什么叫边际？边际就是“新增”带来的“新增”。 例如，边际成本就是每新增一个单位产品所需要付出的新增成本；边际收入是每多卖一个产品能够带来的新增收入；边际产量是每新增一份投入所带来的新增产量；边际效用是每消耗一个单位的商品所能带来的新增享受。
+HL|薛兆丰经济学讲义|薛兆丰|2020/1/30 10:23:58|一个国家很大，贫富有差距，并非每个学校和家长都能负担得起这样标准的校车。标准太高，就会逼着很多学校，尤其是农村的学校放弃提供校车，家长们就只能使用安全性能更低的交通工具，比如自己骑自行车或雇用黑车等，结果是孩子们享受到的安全保障反而降低了。
+NT|薛兆丰经济学讲义|薛兆丰|2020/1/30 10:26:31|山寨 假货 问题

unitest.kman.py

@@ -13,6 +13,8 @@ import json
 from collections import defaultdict
 
 from kman import *
+from parseweb import *
+from mtable import *
 
 # log info
 logger = logging.getLogger()
@@ -31,7 +33,7 @@ class TestKman(unittest.TestCase):
         DELIMITER= '|'
 
         self.km = kMan()
-        self.util = util()
+        self.util = Util()
 
         global t_bm_sec
         global t_hl_sec
@@ -167,7 +169,7 @@ class TestKman(unittest.TestCase):
             print("与预期匹配，sidx 3 重复被删除，抛出: %s" % 'keyerror')
 
         t_secd.clear()
-
+
     # test function format_time()
     def test_format_time(self):
         t_ds = '2020年1月13日 星期一 下午 8:11:05'
@@ -217,7 +219,7 @@ class TestKman(unittest.TestCase):
         logger.debug('========== 2 ==========\n')
         logger.debug(json.dumps(bn, indent=2, ensure_ascii=False))
         bn = {}  
-
+
         # by author
         bn = self.km.filter_clips(t_books, '贾雷德·戴蒙德', 2)
         logger.debug('========== 3 ==========\n')
@@ -228,14 +230,62 @@ class TestKman(unittest.TestCase):
     def test_import_words(self):
         import pprint
         [bookinfo, lookups, words] =  self.km.import_words()
-        # how to beauty print to logger file
+        # how to beauty print to logger file
         logger.debug('========== 4 ==========\n')
         logger.debug(json.dumps(bookinfo, indent=2, ensure_ascii=False))
-        logger.debug('========== 5 ==========\n')
+        logger.debug('========== 5 ==========\n')
         logger.debug(json.dumps(lookups, indent=2, ensure_ascii=False))
         logger.debug('========== 6 ==========\n')
         logger.debug(json.dumps(words, indent=2, ensure_ascii=False))
 
+        # test filter_words()
+        logger.debug('========== 7 ==========\n')
+        self.km.filter_words(self.km.import_words(), '中国历史风云录 (陈舜臣作品)', tp=0)
+
+        logger.debug('========== 8 ==========\n')
+        self.km.filter_words(self.km.import_words(), 'zh:闾', tp=1)
+
+    def test_util(self):
+        logger.debug( 'test get_app_path: {}'.format( self.util.get_app_path()))
+
+    def test_mtable(self):
+        print('\n---------------------')
+        data = [['Ohiox','Ohio','Ohio','Nevada','Nevada'],
+                [2000,2001,2002,2001,2002],
+                [1.5,1.7,3.6,2.4,2.9],
+                [1.5,1.7,3.6,2.4,2.9],
+                ]
+
+        mt = mTable(data,
+                index = ['one', 'two','three','four','five'],
+                columns = ['year','state','pop','debt','xx'])
+
+        print('== frame\n', mt)
+        print('== frame.iat[0, 0]\n', mt.get_iat(0, 0))
+        print('== frame.iat[1, 1]\n', mt.get_iat(1, 1))
+        print('== frame.iat[2, 2]\n', mt.get_iat(2, 2))
+        print('== frame.iat[2, 1]\n', mt.get_iat(2, 1))
+        print('== frame.shape[0]\n', mt.get_num_rows())
+        print('== frame.shape[1]\n', mt.get_num_columns())
+        print('== frame.columns\n', mt.get_columns())
+        print('== frame.index\n', mt.get_index())
+
+        mt = mTable(data,
+                columns = ['year','state','pop','debt','xx'])
+        print('== frame\n', mt)
+        print('== frame.iat[2, 1]\n', mt.get_iat(2, 1))
+        print('== frame.shape[0]\n', mt.get_num_rows())
+        print('== frame.shape[1]\n', mt.get_num_columns())
+        print('== frame.columns\n', mt.get_columns())
+        print('== frame.index\n', mt.get_index())
+
+        mt = mTable(data)
+        mt.set_repr_width(20)
+        print('== frame\n', mt)
+        print('== frame.iat[2, 1]\n', mt.get_iat(2, 1))
+        print('== frame.shape[0]\n', mt.get_num_rows())
+        print('== frame.shape[1]\n', mt.get_num_columns())
+        print('== frame.columns\n', mt.get_columns())
         print('== frame.index\n', mt.get_index())
 
         mt = mTable()
@@ -249,6 +299,80 @@ class TestKman(unittest.TestCase):
     """
     def test_search_clip(self):
         pass
+    def test_statistic(self):
+        pass
+    def test_dict2json(self):
+        pass
+    def test_json2dict(self):
+        pass
+    def test_import_clips(self):
+        pass
+    """
+
+    def test_parseweb(self):
+        spide = bookInfoSpide()
+
+        """
+        for bkname in testbooks:
+            bkname = re.split(r'[\(（\-\:：_\s]',bkname.strip())[0]
+            print(bkname)
+            bkinfo = spide.grab_book_info(bkname)
+            filter_bkinfo = spide.filter_spide_book(bkinfo)
+            if filter_bkinfo: spide.down_book_img(filter_bkinfo)
+
+            logger.debug('================ {} ================'.format(bkname))
+            logger.debug(json.dumps(bkinfo,indent=2, ensure_ascii=False))
+            logger.debug(json.dumps(filter_bkinfo,indent=2, ensure_ascii=False))
+        """
+
+    def test_kman(self):
+        #books = defaultdict(dict)
+        km = kMan()
+        books = km.import_clips()
+
+        # remove duplication
+        km.drop_duplicate(books)
+
+        # test search note function
+        searchnote = km.search_clip(books, '三大都市圈', 'ALL', 'CONTENT')
+        if searchnote[0] > 0: km.export_notes(searchnote[1], 'searchcontent', ft='MD')
+        searchnote = km.search_clip(books, '经济', 'ALL', 'TITLE')
+        if searchnote[0] > 0: km.export_notes(searchnote[1], 'searchtitle', ft='MD')
+        searchnote = km.search_clip(books, '巴曙松', 'ALL', 'AUTHOR')
+        if searchnote[0] > 0: km.export_notes(searchnote[1], 'searchauthor', ft='MD')
+
+        #print(km.get_bookname_num(books))
+        #print(km.get_author_num(books))
+
+        # add note content to hightlight, then delete note
+        km.add_note_to_highlight(books)
+
+        # test dict json convert
+        with open(os.path.join(CURRPATH,'xx'), 'w', encoding='utf8', errors='ignore') as fw:
+            fw.write(km.dict2json(books))
+        if km.json2dict(os.path.join(CURRPATH,'xx'))==books: print( 'test OK')
+
+        km.export_notes(books, OUTPREF, ft='MD')
+
+        # print data with json format
+        logger.debug(json.dumps(books, indent=4, sort_keys=True, ensure_ascii=False))
+
+    def test_somepath(self):
+        frozen = 'not'
+        if getattr(sys, 'frozen', False):
+            # we are running in a bundle
+            frozen = 'ever so'
+            bundle_dir = sys._MEIPASS
+        else:
+            # we are running in a normal Python environment
+            bundle_dir = os.path.dirname(os.path.abspath(__file__))
+        print( 'we are',frozen,'frozen')
+        print( 'bundle dir is', bundle_dir )
+        print( 'sys.argv[0] is', sys.argv[0] )
+        print( 'sys.executable is', sys.executable )
+        print( 'os.getcwd is', os.getcwd() )
+
+        print('sys.path[0]', sys.path[0])
         print('sys.argv[0]', sys.argv[0])
         print('os.path.realpath(sys.executable)', os.path.realpath(sys.executable))
         print('os.path.realpath(sys.argv[0]))', os.path.realpath(sys.argv[0]))
@@ -269,4 +393,3 @@ if __name__ == '__main__':
     suite = unittest.TestSuite ()
     suite.addTest(TestKman('test_parse_section'))
     suite.addTest(TestKman('test_format_time'))
-    suite.addTest(TestKman('test_format_data'))

xx

@@ -0,0 +1 @@
+{"\u859b\u5146\u4e30\u7ecf\u6d4e\u5b66\u8bb2\u4e49": {"author": "\u859b\u5146\u4e30", "1": {"type": "HL", "position": "1408-1410", "day": "2020\u5e741\u670813\u65e5", "week": "\u661f\u671f\u4e00", "meridiem": "\u4e0a\u5348", "time": "8:11:05", "content": "\u4e48\u5230\u5e95\u4ec0\u4e48\u53eb\u8fb9\u9645\uff1f\u8fb9\u9645\u5c31\u662f\u201c\u65b0\u589e\u201d\u5e26\u6765\u7684\u201c\u65b0\u589e\u201d\u3002 \u4f8b\u5982\uff0c\u8fb9\u9645\u6210\u672c\u5c31\u662f\u6bcf\u65b0\u589e\u4e00\u4e2a\u5355\u4f4d\u4ea7\u54c1\u6240\u9700\u8981\u4ed8\u51fa\u7684\u65b0\u589e\u6210\u672c\uff1b\u8fb9\u9645\u6536\u5165\u662f\u6bcf\u591a\u5356\u4e00\u4e2a\u4ea7\u54c1\u80fd\u591f\u5e26\u6765\u7684\u65b0\u589e\u6536\u5165\uff1b\u8fb9\u9645\u4ea7\u91cf\u662f\u6bcf\u65b0\u589e\u4e00\u4efd\u6295\u5165\u6240\u5e26\u6765\u7684\u65b0\u589e\u4ea7\u91cf\uff1b\u8fb9\u9645\u6548\u7528\u662f\u6bcf\u6d88\u8017\u4e00\u4e2a\u5355\u4f4d\u7684\u5546\u54c1\u6240\u80fd\u5e26\u6765\u7684\u65b0\u589e\u4eab\u53d7\u3002"}, "2": {"type": "HL", "position": "4284-4286", "day": "2020\u5e741\u670830\u65e5", "week": "\u661f\u671f\u56db", "meridiem": "\u4e0a\u5348", "time": "10:23:58", "content": "\u4e00\u4e2a\u56fd\u5bb6\u5f88\u5927\uff0c\u8d2b\u5bcc\u6709\u5dee\u8ddd\uff0c\u5e76\u975e\u6bcf\u4e2a\u5b66\u6821\u548c\u5bb6\u957f\u90fd\u80fd\u8d1f\u62c5\u5f97\u8d77\u8fd9\u6837\u6807\u51c6\u7684\u6821\u8f66\u3002\u6807\u51c6\u592a\u9ad8\uff0c\u5c31\u4f1a\u903c\u7740\u5f88\u591a\u5b66\u6821\uff0c\u5c24\u5176\u662f\u519c\u6751\u7684\u5b66\u6821\u653e\u5f03\u63d0\u4f9b\u6821\u8f66\uff0c\u5bb6\u957f\u4eec\u5c31\u53ea\u80fd\u4f7f\u7528\u5b89\u5168\u6027\u80fd\u66f4\u4f4e\u7684\u4ea4\u901a\u5de5\u5177\uff0c\u6bd4\u5982\u81ea\u5df1\u9a91\u81ea\u884c\u8f66\u6216\u96c7\u7528\u9ed1\u8f66\u7b49\uff0c\u7ed3\u679c\u662f\u5b69\u5b50\u4eec\u4eab\u53d7\u5230\u7684\u5b89\u5168\u4fdd\u969c\u53cd\u800c\u964d\u4f4e\u4e86\u3002--CG\u6ce8:\u5c71\u5be8 \u5047\u8d27 \u95ee\u9898"}}, "\u5e86\u4f59\u5e74(\u7cbe\u6821\u7248\uff09": {"author": "\u732b\u817b", "4": {"type": "HL", "position": "48484-48484", "day": "2020\u5e741\u670819\u65e5", "week": "\u661f\u671f\u65e5", "meridiem": "\u4e0b\u5348", "time": "8:00:29", "content": "\u56ed\u5b50\u91cc\u7684\u62a4\u536b\u80fd\u63ba\u591a\u5c11\u4eba\u5c31\u63ba\u591a\u5c11\u4eba\uff0c\u6211\u4f1a\u6d3e\u4eba\u76ef\u7740"}, "5": {"type": "HL", "position": "49901-49901", "day": "2020\u5e741\u670820\u65e5", "week": "\u661f\u671f\u4e00", "meridiem": "\u4e0b\u5348", "time": "7:57:10", "content": "\u53f6\u7075\u513f\u53f9\u4e86\u53e3"}}}