{"id":1803,"date":"2017-05-31T10:30:43","date_gmt":"2017-05-31T01:30:43","guid":{"rendered":"https:\/\/blog.ymyzk.com\/?p=1803"},"modified":"2017-05-30T22:05:31","modified_gmt":"2017-05-30T13:05:31","slug":"pypi-download-stats-bigquery","status":"publish","type":"post","link":"https:\/\/blog.ymyzk.com\/2017\/05\/pypi-download-stats-bigquery\/","title":{"rendered":"PyPI \u306e\u7d71\u8a08\u60c5\u5831\u3092 BigQuery \u3067\u96c6\u8a08\u3059\u308b"},"content":{"rendered":"<p>Python \u306e\u30d1\u30c3\u30b1\u30fc\u30b8\u30a4\u30f3\u30c7\u30c3\u30af\u30b9 <a href=\"https:\/\/pypi.python.org\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">PyPI<\/a> \u306b\u304a\u3051\u308b\uff0c\u30d1\u30c3\u30b1\u30fc\u30b8\u306e\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u6570\u7b49\u306e\u30c7\u30fc\u30bf\u306f BigQuery \u3067\u516c\u958b\u3055\u308c\u3066\u3044\u307e\u3059\uff0e\u3053\u306e\u30c7\u30fc\u30bf\u3092\u5229\u7528\u3059\u308b\u3068\u30d1\u30c3\u30b1\u30fc\u30b8\u306e\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u6570\u3092\u96c6\u8a08\u3057\u305f\u308a\uff0c\u3088\u304f\u4f7f\u308f\u308c\u3066\u3044\u308b\u51e6\u7406\u7cfb\u306e\u30d0\u30fc\u30b8\u30e7\u30f3\u306a\u3069\u3092\u7c21\u5358\u306b\u77e5\u308b\u3053\u3068\u304c\u51fa\u6765\u307e\u3059\uff0e<\/p>\n<p><!--more--><\/p>\n<h2>\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8<\/h2>\n<p>PyPI \u306e\u7d71\u8a08\u60c5\u5831\u306e\u30c7\u30fc\u30bf\u30bb\u30c3\u30c8\u306f <a href=\"https:\/\/bigquery.cloud.google.com\/dataset\/the-psf:pypi\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">the-psd:pypi<\/a> \u3067\u516c\u958b\u3055\u308c\u3066\u3044\u307e\u3059\uff0e\u65e5\u3054\u3068\u306b\u30c6\u30fc\u30d6\u30eb\u304c\u4f5c\u6210\u3055\u308c\u3066\u304a\u308a\uff0cdownloadsYYYYMMDD \u306e\u3088\u3046\u306a\u30c6\u30fc\u30d6\u30eb\u540d\u306b\u306a\u3063\u3066\u3044\u307e\u3059\uff0e1\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u3054\u3068\u306b1\u884c\u306e\u30c7\u30fc\u30bf\u306b\u306a\u3063\u3066\u304a\u308a\uff0c\u4ee5\u4e0b\u306e\u3088\u3046\u306a\u69d8\u3005\u306a\u60c5\u5831\u304c\u8a18\u9332\u3055\u308c\u3066\u3044\u307e\u3059\uff0e<\/p>\n<ul>\n<li>\u30bf\u30a4\u30e0\u30b9\u30bf\u30f3\u30d7<\/li>\n<li>\u56fd\u30b3\u30fc\u30c9<\/li>\n<li>\u30d7\u30ed\u30b8\u30a7\u30af\u30c8\u540d (\u30d1\u30c3\u30b1\u30fc\u30b8\u540d)<\/li>\n<li>\u30d0\u30fc\u30b8\u30e7\u30f3<\/li>\n<li>\u5f62\u5f0f (sdist\u30fbbdist_wheel \u306a\u3069)<\/li>\n<li>OS \u306e\u60c5\u5831<\/li>\n<li>\u51e6\u7406\u7cfb\u306e\u60c5\u5831<\/li>\n<li>TLS \u95a2\u4fc2\u306e\u60c5\u5831<\/li>\n<\/ul>\n<h2>\u30c7\u30fc\u30bf\u3092\u96c6\u8a08\u3059\u308b<\/h2>\n<p>\u4f8b\u3048\u3070\u3042\u308b\u65e5\u306e\u3059\u3079\u3066\u306e\u30d1\u30c3\u30b1\u30fc\u30b8\u306e\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u6570\u306e\u5408\u8a08\u306f\u6b21\u306e\u3088\u3046\u306b\u3057\u3066\u96c6\u8a08\u3067\u304d\u307e\u3059\uff0e<\/p>\n<pre><code class=\"sql\">SELECT\r\n  COUNT(*) AS download_count,\r\nFROM\r\n  [the-psf:pypi.downloads20170529]\r\nORDER BY\r\n  download_count DESC<\/code><\/pre>\n<p>\u7d50\u679c\u306f\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u306a\u308a\u307e\u3059\uff0e\u7d042,257\u4e07\u56de\u3082\u30d1\u30c3\u30b1\u30fc\u30b8\u304c\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u3055\u308c\u3066\u3044\u308b\u3088\u3046\u3067\u3059\uff0e<\/p>\n<pre><code class=\"json\">[\r\n  {\r\n    \"download_count\": \"22573815\"\r\n  }\r\n]<\/code><\/pre>\n<p>\u6709\u540d\u306a\u30d1\u30c3\u30b1\u30fc\u30b8\u3067\u3042\u308b requests \u306e\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u6570\u3092\u96c6\u8a08\u3059\u308b\u3068\uff0c\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u306a\u308a\u307e\u3059\uff0e<\/p>\n<pre><code class=\"sql\">SELECT\r\n  COUNT(*) AS download_count,\r\nFROM\r\n  [the-psf:pypi.downloads20170529]\r\nWHERE\r\n  file.project = \"requests\"\r\nORDER BY\r\n  download_count DESC<\/code><\/pre>\n<p>\u7d0444\u4e07\u56de\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u3055\u308c\u3066\u3044\u308b\u3088\u3046\u3067\u3059\uff0e<\/p>\n<pre><code class=\"json\">[\r\n  {\r\n    \"download_count\": \"445065\"\r\n  }\r\n]<\/code><\/pre>\n<p>(\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u306b\u4f7f\u7528\u3055\u308c\u305f) Python \u306e\u30d0\u30fc\u30b8\u30e7\u30f3\u6bce\u306e\uff0c\u30d1\u30c3\u30b1\u30fc\u30b8\u306e\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u6570\u3082\u96c6\u8a08\u3067\u304d\u307e\u3059\uff0e<\/p>\n<pre><code class=\"sql\">SELECT\r\n  REGEXP_EXTRACT(details.python, r\"^([^\\.]+\\.[^\\.]+)\") as python_version,\r\n  COUNT(*) as download_count,\r\nFROM\r\n  [the-psf:pypi.downloads20170529]\r\nGROUP BY\r\n  python_version,\r\nORDER BY\r\n  download_count DESC<\/code><\/pre>\n<p>\u7d50\u679c\u306f\u4ee5\u4e0b\u306e\u3088\u3046\u306b\u306a\u308a\u307e\u3059\uff0e<\/p>\n<pre><code class=\"json\">[\r\n  {\r\n    \"python_version\": \"2.7\",\r\n    \"download_count\": \"16162589\"\r\n  },\r\n  {\r\n    \"python_version\": \"3.5\",\r\n    \"download_count\": \"2249664\"\r\n  },\r\n  {\r\n    \"python_version\": null,\r\n    \"download_count\": \"1770785\"\r\n  },\r\n  {\r\n    \"python_version\": \"3.4\",\r\n    \"download_count\": \"897013\"\r\n  },\r\n  {\r\n    \"python_version\": \"3.6\",\r\n    \"download_count\": \"888574\"\r\n  },\r\n  {\r\n    \"python_version\": \"2.6\",\r\n    \"download_count\": \"527765\"\r\n  },\r\n  {\r\n    \"python_version\": \"3.3\",\r\n    \"download_count\": \"58497\"\r\n  },\r\n  {\r\n    \"python_version\": \"3.7\",\r\n    \"download_count\": \"11959\"\r\n  },\r\n  {\r\n    \"python_version\": \"3.2\",\r\n    \"download_count\": \"5401\"\r\n  },\r\n  {\r\n    \"python_version\": \"1.17\",\r\n    \"download_count\": \"1369\"\r\n  },\r\n  {\r\n    \"python_version\": \"2.4\",\r\n    \"download_count\": \"121\"\r\n  },\r\n  {\r\n    \"python_version\": \"2.5\",\r\n    \"download_count\": \"76\"\r\n  }\r\n]<\/code><\/pre>\n<p>\u7d041600\u4e07\u3067\uff0cPython 2.7 \u304c\u5727\u5012\u7684\u306a\u512a\u52dd\u3067\u3059\ud83d\ude2d\ud83d\ude2d\ud83d\ude2d\uff0ePython 3.5 \u306f \u7d04224\u4e07\u30672\u4f4d\u306b\u306a\u3063\u3066\u3044\u307e\u3059\uff0e<\/p>\n<p>Python 3.x \u7cfb\u306f\u30de\u30a4\u30ca\u30fc\u30d0\u30fc\u30b8\u30e7\u30f3\u3067\u7968\u304c\u5272\u308c\u3066\u3044\u308b\u306e\u3067\u3042\u307e\u308a\u30d5\u30a7\u30a2\u306a\u6bd4\u8f03\u3067\u306f\u306a\u3044\u304b\u3082\u3057\u308c\u307e\u305b\u3093\uff0e\u3061\u306a\u307f\u306b\uff0cPython 3.x \u306e\u7d50\u679c\u3092\u8db3\u3057\u3042\u308f\u3059\u3068\u7d04410\u4e07\u306b\u306a\u308b\u306e\u3067\uff0cPython 2 \u3068 3 \u306e\u6bd4\u7387\u306f 4:1 \u304f\u3089\u3044\u306b\u306a\u308a\u307e\u3059\uff0e<\/p>\n<h2>pypinfo \u3067\u3088\u308a\u7c21\u5358\u306b\u96c6\u8a08\u3059\u308b<\/h2>\n<p>\u5b9f\u306f\u4e0a\u306b\u6319\u3052\u305f\u30af\u30a8\u30ea\u306f\uff0c<a href=\"https:\/\/github.com\/ofek\/pypinfo\" target=\"_blank\" rel=\"noopener noreferrer nofollow\">pypinfo<\/a> \u3068\u3044\u3046\u30c4\u30fc\u30eb\u3092\u4f7f\u3063\u3066\u4f5c\u3063\u305f\u3082\u306e\u3092\u4fee\u6b63\u3057\u305f\u3082\u306e\u3067\u3059\uff0epypinfo \u3092\u4f7f\u3046\u3068 CLI \u304b\u3089\u7c21\u5358\u306b\u96c6\u8a08\u3092\u884c\u3063\u305f\u308a\uff0c\u30af\u30a8\u30ea\u3092\u751f\u6210\u3057\u305f\u308a\u3059\u308b\u3053\u3068\u304c\u51fa\u6765\u307e\u3059\uff0e\u8a73\u7d30\u306f pypinfo \u306e README \u3092\u53c2\u7167\u3057\u3066\u4e0b\u3055\u3044\uff0e\u4f8b:<\/p>\n<pre><code class=\"custom-plain\">$ pypinfo requests\r\ndownload_count\r\n--------------\r\n11033343<\/code><\/pre>\n<h2>\u307e\u3068\u3081<\/h2>\n<p>BigQuery \u3092\u4f7f\u3046\u3068\uff0cPyPI \u306e\u7d71\u8a08\u60c5\u5831\u3092\u7c21\u5358\u306b\u96c6\u8a08\u3059\u308b\u3053\u3068\u304c\u51fa\u6765\uff0c\u81ea\u5206\u306e\u4f5c\u3063\u305f\u30d1\u30c3\u30b1\u30fc\u30b8\u306e\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u6570\u306a\u3069\u3082\u7c21\u5358\u306b\u8abf\u3079\u308b\u3053\u3068\u304c\u51fa\u6765\u307e\u3059\uff0e\u307e\u305f\uff0cpypinfo \u3092\u4f7f\u3046\u3068\u3088\u308a\u7c21\u5358\u306b\u96c6\u8a08\u3092\u884c\u3046\u3053\u3068\u304c\u53ef\u80fd\u3067\u3059\uff0e<\/p>\n<p>\u30c6\u30fc\u30d6\u30eb\u306e\u30b5\u30a4\u30ba\u304c\u975e\u5e38\u306b\u5927\u304d\u3044\u306e\u3067\uff0c\u8ab2\u91d1\u306b\u6ce8\u610f\u3057\u307e\u3057\u3087\u3046\uff0e(\u6bce\u67081TB\u307e\u3067\u306e\u30c7\u30fc\u30bf\u306e\u30af\u30a8\u30ea\u306f\u7121\u6599\u3067\u3059\uff0e) \u4f8b\u3048\u3070 2017\/5\/28 \u306e\u30c6\u30fc\u30d6\u30eb\u306e\u30b5\u30a4\u30ba\u306f\u7d045GB\u3042\u308a\u307e\u3059\uff0e\u5fc5\u8981\u306a\u30ab\u30e9\u30e0\u306e\u30c7\u30fc\u30bf\u306e\u307f\u3092\u53d6\u5f97\u3059\u308b\u3088\u3046\u306b\u3059\u308c\u30701\u56de\u306e\u30af\u30a8\u30ea\u5b9f\u884c\u3067\u8aad\u307f\u8fbc\u3080\u30c7\u30fc\u30bf\u306f\u6570\u767eMB\u7a0b\u5ea6\u306b\u306a\u308b\u306f\u305a\u3067\u3059\uff0e<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Python \u306e\u30d1\u30c3\u30b1\u30fc\u30b8\u30a4\u30f3\u30c7\u30c3\u30af\u30b9 PyPI \u306b\u304a\u3051\u308b\uff0c\u30d1\u30c3\u30b1\u30fc\u30b8\u306e\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u6570\u7b49\u306e\u30c7\u30fc\u30bf\u306f BigQuery \u3067\u516c\u958b\u3055\u308c\u3066\u3044\u307e\u3059\uff0e\u3053\u306e\u30c7\u30fc\u30bf\u3092\u5229\u7528\u3059\u308b\u3068\u30d1\u30c3\u30b1\u30fc\u30b8\u306e\u30c0\u30a6\u30f3\u30ed\u30fc\u30c9\u6570\u3092\u96c6\u8a08\u3057\u305f\u308a\uff0c\u3088\u304f\u4f7f\u308f\u308c\u3066\u3044\u308b\u51e6\u7406 [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":1805,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[26],"tags":[98,15],"_links":{"self":[{"href":"https:\/\/blog.ymyzk.com\/wp-json\/wp\/v2\/posts\/1803"}],"collection":[{"href":"https:\/\/blog.ymyzk.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.ymyzk.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.ymyzk.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.ymyzk.com\/wp-json\/wp\/v2\/comments?post=1803"}],"version-history":[{"count":4,"href":"https:\/\/blog.ymyzk.com\/wp-json\/wp\/v2\/posts\/1803\/revisions"}],"predecessor-version":[{"id":1808,"href":"https:\/\/blog.ymyzk.com\/wp-json\/wp\/v2\/posts\/1803\/revisions\/1808"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.ymyzk.com\/wp-json\/wp\/v2\/media\/1805"}],"wp:attachment":[{"href":"https:\/\/blog.ymyzk.com\/wp-json\/wp\/v2\/media?parent=1803"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.ymyzk.com\/wp-json\/wp\/v2\/categories?post=1803"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.ymyzk.com\/wp-json\/wp\/v2\/tags?post=1803"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}