{ "cells": [ { "cell_type": "markdown", "id": "8b22a2d1", "metadata": {}, "source": [ "# Introduction\n", "\n", "Bonjour et bienvenue dans le notebook du projet sur les bases de données tablebase.\n", "\n", "Dans ce notebook. Nous allons montrer comme fonctionne le stockage d'une base de données tablebases en prenant comme exemple la table base \"syzygy\" avec comme taille 3 à 5 pièces pour des raisons de taille.\n", "Nous allons expliquer comment les finales ( ou position d'échecs) sont stockés dans le format key-value puis nous verrons ensuite les performances de la base à l'aide de Redis.\n" ] }, { "cell_type": "code", "execution_count": 63, "id": "78fdf40a", "metadata": { "ExecuteTime": { "end_time": "2025-11-21T19:39:06.013699Z", "start_time": "2025-11-21T19:39:05.633099Z" } }, "outputs": [], "source": [ "#import du projet\n", "import redis\n", "import time\n", "import chess\n", "import chess.syzygy\n", "import random\n", "import numpy as np\n", "import pandas as pd\n", "import os\n", "from IPython.display import display\n", "from scipy import stats\n" ] }, { "cell_type": "markdown", "id": "d584aba1", "metadata": {}, "source": [ "## Peuplement de Redis\n", "\n", "Dans un premier temps, on ajoute les positions de la tablebase à Redis afin de pouvoir les utilisers derrières pour faire les tests." ] }, { "cell_type": "code", "id": "338f123d", "metadata": { "ExecuteTime": { "end_time": "2025-11-21T20:10:17.877790Z", "start_time": "2025-11-21T20:10:07.868909Z" } }, "source": [ "#Fonction pour prendre en main Redis\n", "#Connect to Redis server\n", "\n", "def do_this():\n", "\n", " redis_server = redis.Redis(host='localhost', port=6379, db=0)\n", "\n", " #Set and get a simple key-value pair \n", "\n", " redis_server.set('example_key', 'example_value')\n", " value = redis_server.get('example_key')\n", "\n", " print(\"Value set. Retrieved from Redis:\", value.decode('utf-8'))\n", "\n", " count = 1\n", " while count <= 10:\n", " print(f\"Time == {count} seconds\")\n", " value = redis_server.get('example_key')\n", " print(f\"Retrieved from Redis: {value}\")\n", " print(f\"Retrieved from Redis: {redis_server.get(name = 'exemple2')}\")\n", "\n", " count +=1\n", " time.sleep(1)\n", " \n", "if __name__ == \"__main__\":\n", " do_this()" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Value set. Retrieved from Redis: example_value\n", "Time == 1 seconds\n", "Retrieved from Redis: b'example_value'\n", "Retrieved from Redis: None\n", "Time == 2 seconds\n", "Retrieved from Redis: b'example_value'\n", "Retrieved from Redis: None\n", "Time == 3 seconds\n", "Retrieved from Redis: b'example_value'\n", "Retrieved from Redis: None\n", "Time == 4 seconds\n", "Retrieved from Redis: b'example_value'\n", "Retrieved from Redis: None\n", "Time == 5 seconds\n", "Retrieved from Redis: b'example_value'\n", "Retrieved from Redis: None\n", "Time == 6 seconds\n", "Retrieved from Redis: b'example_value'\n", "Retrieved from Redis: None\n", "Time == 7 seconds\n", "Retrieved from Redis: b'example_value'\n", "Retrieved from Redis: None\n", "Time == 8 seconds\n", "Retrieved from Redis: b'example_value'\n", "Retrieved from Redis: None\n", "Time == 9 seconds\n", "Retrieved from Redis: b'example_value'\n", "Retrieved from Redis: None\n", "Time == 10 seconds\n", "Retrieved from Redis: b'example_value'\n", "Retrieved from Redis: None\n" ] } ], "execution_count": 71 }, { "cell_type": "code", "execution_count": 65, "id": "e6058974", "metadata": { "ExecuteTime": { "end_time": "2025-11-21T19:21:40.515403Z", "start_time": "2025-11-21T19:21:39.200072Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Configs ok: {'KQvKP', 'KPvK', 'KRvK', 'KQvK', 'KBNvK', 'KRvKB', 'KRPvKP', 'KRBvKP', 'KPvKP'}\n" ] }, { "data": { "image/svg+xml": [ "
. . . . k . . .\n",
       ". . . . . . . .\n",
       ". . . n . . . .\n",
       ". . . . . . . .\n",
       ". . . . . . . p\n",
       ". P . . . . . .\n",
       ". . . . B . . .\n",
       ". . . . K . . .
" ], "text/plain": [ "Board('4k3/8/3n4/8/7p/1P6/4B3/4K3 b - - 0 1')" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Récupération des positions de la tablebase KBNvK\n", "Récupération des positions de la tablebase KPvK\n", "Récupération des positions de la tablebase KPvKP\n", "Récupération des positions de la tablebase KQvK\n", "Récupération des positions de la tablebase KQvKP\n", "Récupération des positions de la tablebase KRBvKP\n", "Récupération des positions de la tablebase KRPvKP\n", "Récupération des positions de la tablebase KRvK\n", "Récupération des positions de la tablebase KRvKB\n", "Total positions récupérées: 2250\n", "Ajoute de toutes les positions des tablebases dans Redis terminé!\n" ] } ], "source": [ "def get_250_random_position(tablebases, config):\n", " positions = []\n", " tries = 0\n", "\n", " while len(positions) < 250 and tries < 5000:\n", " board = generate_board_from_config(config)\n", " try:\n", " tablebases.probe_wdl(board) # thử probe WDL\n", " positions.append(board)\n", " except chess.syzygy.MissingTableError:\n", " pass # bỏ qua nếu table không tồn tại\n", " tries += 1\n", " return positions\n", "\n", "def generate_board_from_config(config):\n", " \"\"\"\n", " Génère un board contenant exactement le matériel d'une config Syzygy. Exemples de config :\n", " - 'KQvK', 'KPvKP'\n", " Cette fonction est important car il faut générer les positions qui dont les informations sont dans la tablebases.\n", " \"\"\"\n", "\n", " piece_map = {\"K\": chess.KING,\"Q\": chess.QUEEN,\"R\": chess.ROOK,\"B\": chess.BISHOP,\"N\": chess.KNIGHT,\"P\": chess.PAWN}\n", "\n", " white_str, black_str = config.split(\"v\") #Sépare la config entre les 2 couleurs\n", "\n", " board = chess.Board(None) # donne un plateau vide\n", "\n", " # boucles pour placer les pièces sur le plateau\n", " for p in white_str:\n", " piece = piece_map[p]\n", " sq = random.choice([s for s in chess.SQUARES if board.piece_at(s) is None]) #choisi une pièce et vérifie que il n'y a pas déjà une pièce dessus\n", " board.set_piece_at(sq, chess.Piece(piece, chess.WHITE))\n", "\n", " for p in black_str:\n", " piece = piece_map[p]\n", " sq = random.choice([s for s in chess.SQUARES if board.piece_at(s) is None])#choisi une pièce et vérifie que il n'y a pas déjà une pièce dessus\n", " board.set_piece_at(sq, chess.Piece(piece, chess.BLACK))\n", "\n", " # désactive les règles des échecs.\n", " board.castling_rights = 0\n", " board.ep_square = None\n", "\n", " # Donne une couleur aléatoire pour savoir quelle couleur doit jouer le prochain coup.\n", " board.turn = random.choice([chess.WHITE, chess.BLACK])\n", "\n", " # Vérifier que la position est légale pour \n", " if not board.is_valid():\n", " return generate_board_from_config(config) # regénère si la position n'est pas valide\n", "\n", " return board\n", "\n", "def add_tablebase_to_redis():\n", " #Connect to Redis server\n", " redis_server = redis.Redis(host='localhost', port=6379, db=0)\n", " \n", " tablebases_path = \"tablebases/\" #Chemin vers les tables bases Syzygy\n", "\n", " tablebases = chess.syzygy.open_tablebase(tablebases_path)\n", "\n", " available_tables = set()\n", " for f in os.listdir(tablebases_path):\n", " if f.endswith(\".rtbw\") or f.endswith(\".rtbz\"):\n", " available_tables.add(f.split(\".\")[0])\n", "\n", " print(\"Configs ok:\", available_tables)\n", "\n", " position = chess.Board(\"4k3/8/3n4/8/7p/1P6/4B3/4K3 b - - 0 1\") #Position d'exemple\n", " display(position) \n", " \n", " configs = [\"KBNvK\",\"KPvK\",\"KPvKP\",\"KQvK\",\"KQvKP\",\"KRBvKP\",\"KRPvKP\",\"KRvK\",\"KRvKB\"]\n", " config_valide = [c for c in configs if c in available_tables]\n", " #Configurations de toute les tablebases que l'on utilisera dans le projet.\n", " all_positions = []\n", " for config in config_valide:\n", " print(f\"Récupération des positions de la tablebase {config}\")\n", " position = get_250_random_position(tablebases,config)\n", " all_positions.extend(position) #Ajout des positions générées à la liste globale\n", " \n", " print(f\"Total positions récupérées: {len(all_positions)}\")\n", " for position in all_positions:\n", " try:\n", " fen = position.fen()\n", " wdl = tablebases.probe_wdl(position)\n", " dtz = tablebases.probe_dtz(position)\n", " redis_server.hset(fen, mapping={\"wdl\": wdl, \"dtz\": dtz}) \n", " except (KeyError,chess.syzygy.MissingTableError): #vérifie que les positions générés appartiennent bien au table du projet seulement\n", " continue \n", " \n", " print(\"Ajoute de toutes les positions des tablebases dans Redis terminé!\")\n", "\n", "if __name__ == \"__main__\":\n", " add_tablebase_to_redis() " ] }, { "cell_type": "markdown", "id": "1b443c8c", "metadata": {}, "source": [ "## Accès à Redis\n", "\n", "Nous avons rajouter 250 positions apparentant à chacune des tablesbases présent dans le dossier. Il y a des configurations ayant 3,4 ainsi que 5 pièces pour voir si cela a une incidence sur le temps d'accès à Redis.\n", "\n", "Pour la suite, nous allons donc pouvoir passer sur Redis pour vérifier cette hypothèse et faire d'autres tests sur notre base.\n", "\n", "## Benchmark redis" ] }, { "cell_type": "code", "execution_count": 66, "id": "8d49b928-7522-4727-b6d8-224e2bd3a1b7", "metadata": { "ExecuteTime": { "end_time": "2025-11-21T19:48:52.738372Z", "start_time": "2025-11-21T19:48:52.733758Z" } }, "outputs": [], "source": [ "def benchmark_redis(redis_serveur, pos):\n", " latency = []\n", " start_time_all = time.time()\n", "\n", " for board in pos:\n", " fen = board.fen()\n", " start_individual = time.time()\n", " redis_serveur.hset(fen, mapping={\"board\": board.fen()})\n", " latency.append(time.time() - start_individual)\n", "\n", " total_time_all=time.time()-start_time_all\n", " latency= np.array(latency)\n", "\n", " return{\n", " \"count\": len(pos),\n", " \"average_latency\": latency.mean(),\n", " \"p50\": np.percentile(latency,50),\n", " \"p90\": np.percentile(latency,90),\n", " \"p95\": np.percentile(latency,95),\n", " \"p99\": np.percentile(latency,99),\n", " \"std\": latency.std(),\n", " \"min\": latency.min(),\n", " \"max\": latency.max(),\n", " \"total_time\": total_time_all,\n", " \"latency\": latency,\n", " \"rps\": len(latency)/total_time_all,\n", " }\n", "\n", "def benchmark_syzygy(tablebases, pos):\n", " latency = []\n", " start_time_all = time.time()\n", "\n", " for board in pos:\n", " try:\n", " start = time.time()\n", " tablebases.probe_wdl(board)\n", " tablebases.probe_dtz(board)\n", " latency.append(time.time() - start)\n", " except chess.syzygy.MissingTableError:\n", " continue\n", "\n", " total_time_all=time.time()-start_time_all\n", "\n", " return{\n", " \"count\": len(pos),\n", " \"average_latency\": np.mean(latency),\n", " \"p50\": np.percentile(latency,50),\n", " \"p90\": np.percentile(latency,90),\n", " \"p95\": np.percentile(latency,95),\n", " \"p99\": np.percentile(latency,99),\n", " \"std\": np.std(latency),\n", " \"min\": np.min(latency),\n", " \"max\": np.max(latency),\n", " \"total_time\": total_time_all,\n", " \"latency\": latency,\n", " \"rps\": len(latency)/total_time_all,\n", " }\n", "\n", "def get_redis_memory(redis_server):\n", " info = redis_server.info(\"memory\")\n", " return info[\"used_memory\"], info[\"used_memory_human\"]\n" ] }, { "cell_type": "code", "execution_count": 70, "id": "0d955b0d-dc80-4518-954b-7c2df295adca", "metadata": { "ExecuteTime": { "end_time": "2025-11-21T19:48:34.897816Z", "start_time": "2025-11-21T19:48:33.514880Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Redis latency array length: 2250\n", "Syzygy latency array length: 2011\n", "BENCHMARK SYZYGY\n", "{'count': 2250, 'average_latency': np.float64(0.00036636051253892887), 'p50': np.float64(0.00013303756713867188), 'p90': np.float64(0.0006931543350219727), 'p95': np.float64(0.000935828685760498), 'p99': np.float64(0.004939346313476563), 'std': np.float64(0.0007832406414951507), 'min': np.float64(2.2411346435546875e-05), 'max': np.float64(0.007592201232910156), 'total_time': 0.8658432960510254, 'latency': [0.005835056304931641, 0.0063092708587646484, 4.0531158447265625e-05, 0.005937337875366211, 0.004686594009399414, 0.004458904266357422, 0.005086660385131836, 0.003803730010986328, 0.0001289844512939453, 0.0033648014068603516, 0.00040411949157714844, 0.004712820053100586, 0.0003044605255126953, 3.600120544433594e-05, 0.0002243518829345703, 0.00011539459228515625, 0.00016760826110839844, 0.0004913806915283203, 0.0004279613494873047, 0.0001709461212158203, 0.00028705596923828125, 0.00020933151245117188, 0.0003590583801269531, 0.00048828125, 0.0003910064697265625, 4.553794860839844e-05, 0.00038170814514160156, 0.0006213188171386719, 0.00025343894958496094, 0.0004963874816894531, 0.0005056858062744141, 0.00016570091247558594, 0.004057884216308594, 0.0004956722259521484, 0.005513429641723633, 0.0034971237182617188, 0.0001938343048095703, 0.00441741943359375, 0.0003509521484375, 0.00041413307189941406, 0.005630016326904297, 0.00038313865661621094, 0.0034050941467285156, 0.00036072731018066406, 0.00023221969604492188, 0.007534980773925781, 0.0002269744873046875, 0.00040721893310546875, 0.0003025531768798828, 0.00041294097900390625, 0.00028824806213378906, 0.00031566619873046875, 0.0002522468566894531, 0.0004901885986328125, 0.003751039505004883, 0.00043082237243652344, 0.0003578662872314453, 0.004171133041381836, 0.00047659873962402344, 0.0004544258117675781, 0.00011086463928222656, 0.00017189979553222656, 0.004932403564453125, 0.0004432201385498047, 0.0004925727844238281, 0.0003731250762939453, 0.00024127960205078125, 0.005918741226196289, 0.0004565715789794922, 0.005646228790283203, 0.00044345855712890625, 0.0002887248992919922, 5.221366882324219e-05, 0.006527900695800781, 0.006245136260986328, 0.005795955657958984, 0.007592201232910156, 0.00021505355834960938, 0.0002529621124267578, 0.0001819133758544922, 0.005423307418823242, 0.00015807151794433594, 0.0004038810729980469, 0.00014019012451171875, 0.00010371208190917969, 0.000255584716796875, 0.00025844573974609375, 0.0003707408905029297, 0.0004951953887939453, 0.006442070007324219, 0.00017833709716796875, 0.00032901763916015625, 0.005776166915893555, 0.00019598007202148438, 0.00024199485778808594, 0.004564523696899414, 0.0043795108795166016, 0.004447221755981445, 0.0002079010009765625, 0.0002090930938720703, 0.0003058910369873047, 0.00033736228942871094, 0.0038650035858154297, 0.0001900196075439453, 0.0003292560577392578, 0.00414586067199707, 0.00021529197692871094, 0.0033817291259765625, 0.00023031234741210938, 0.00025343894958496094, 0.0003025531768798828, 0.0036134719848632812, 0.00013828277587890625, 9.34600830078125e-05, 0.0003120899200439453, 0.00024056434631347656, 0.00019812583923339844, 0.00023651123046875, 0.005423307418823242, 0.003107786178588867, 0.00031447410583496094, 0.00013709068298339844, 0.0025718212127685547, 0.00028395652770996094, 0.00017690658569335938, 7.200241088867188e-05, 0.0036039352416992188, 4.982948303222656e-05, 0.004940032958984375, 0.0003342628479003906, 0.0001652240753173828, 0.00010251998901367188, 7.843971252441406e-05, 0.0003235340118408203, 0.0002968311309814453, 0.003137350082397461, 0.0004086494445800781, 0.00047326087951660156, 0.0002570152282714844, 0.0004220008850097656, 0.0001926422119140625, 9.655952453613281e-05, 0.00010442733764648438, 0.0001857280731201172, 0.00012755393981933594, 0.0050313472747802734, 0.00044798851013183594, 0.003897428512573242, 0.003752470016479492, 0.004637241363525391, 0.0001685619354248047, 0.0027260780334472656, 0.0004324913024902344, 0.00014066696166992188, 0.0003159046173095703, 0.0036246776580810547, 0.004236936569213867, 0.002907276153564453, 0.00027489662170410156, 0.0003941059112548828, 0.0034711360931396484, 0.00033664703369140625, 0.00031280517578125, 0.00017452239990234375, 0.0001347064971923828, 0.00031113624572753906, 0.00012612342834472656, 0.00027298927307128906, 0.0001347064971923828, 0.0031452178955078125, 8.273124694824219e-05, 0.003012418746948242, 6.67572021484375e-05, 0.00020313262939453125, 0.00016951560974121094, 0.00024390220642089844, 0.00015497207641601562, 0.0007174015045166016, 7.343292236328125e-05, 9.942054748535156e-05, 2.6464462280273438e-05, 0.00031185150146484375, 4.0531158447265625e-05, 0.00021457672119140625, 4.601478576660156e-05, 0.00012302398681640625, 6.437301635742188e-05, 0.00067901611328125, 0.0005671977996826172, 9.131431579589844e-05, 0.00010442733764648438, 6.246566772460938e-05, 0.00021529197692871094, 4.7206878662109375e-05, 0.0005750656127929688, 7.677078247070312e-05, 6.4849853515625e-05, 7.486343383789062e-05, 8.034706115722656e-05, 0.00022459030151367188, 0.00040602684020996094, 6.842613220214844e-05, 3.4809112548828125e-05, 6.866455078125e-05, 4.744529724121094e-05, 8.797645568847656e-05, 4.982948303222656e-05, 6.747245788574219e-05, 8.034706115722656e-05, 4.3392181396484375e-05, 0.0003123283386230469, 5.888938903808594e-05, 0.00030159950256347656, 0.0005438327789306641, 3.9577484130859375e-05, 0.0006866455078125, 9.036064147949219e-05, 4.76837158203125e-05, 0.00048065185546875, 0.00012135505676269531, 4.696846008300781e-05, 0.0001285076141357422, 0.0003001689910888672, 5.14984130859375e-05, 5.53131103515625e-05, 7.343292236328125e-05, 5.364418029785156e-05, 6.723403930664062e-05, 7.62939453125e-05, 0.00022459030151367188, 4.38690185546875e-05, 8.58306884765625e-05, 7.152557373046875e-05, 4.7206878662109375e-05, 5.0067901611328125e-05, 7.343292236328125e-05, 9.083747863769531e-05, 5.2928924560546875e-05, 6.29425048828125e-05, 2.3126602172851562e-05, 4.458427429199219e-05, 2.765655517578125e-05, 4.982948303222656e-05, 6.151199340820312e-05, 0.000457763671875, 0.0005977153778076172, 0.0004870891571044922, 7.772445678710938e-05, 0.0005505084991455078, 6.127357482910156e-05, 5.9604644775390625e-05, 0.00023937225341796875, 0.00010609626770019531, 0.0005023479461669922, 7.677078247070312e-05, 5.602836608886719e-05, 0.00026106834411621094, 6.031990051269531e-05, 0.000385284423828125, 4.887580871582031e-05, 0.00028395652770996094, 3.0517578125e-05, 0.000335693359375, 4.410743713378906e-05, 6.175041198730469e-05, 7.867813110351562e-05, 8.034706115722656e-05, 0.00047087669372558594, 7.414817810058594e-05, 0.0006930828094482422, 0.00028014183044433594, 6.842613220214844e-05, 5.698204040527344e-05, 0.000244140625, 0.0004906654357910156, 0.00023174285888671875, 0.00043320655822753906, 5.173683166503906e-05, 9.226799011230469e-05, 3.2901763916015625e-05, 0.00044226646423339844, 0.0001373291015625, 0.0006694793701171875, 0.0005078315734863281, 3.170967102050781e-05, 0.0005376338958740234, 0.0005660057067871094, 0.00010752677917480469, 0.00038051605224609375, 8.654594421386719e-05, 4.363059997558594e-05, 0.0005552768707275391, 5.1021575927734375e-05, 4.291534423828125e-05, 3.314018249511719e-05, 6.127357482910156e-05, 5.078315734863281e-05, 0.00010561943054199219, 5.936622619628906e-05, 0.00030875205993652344, 0.00010895729064941406, 0.00035858154296875, 0.0003674030303955078, 5.054473876953125e-05, 5.030632019042969e-05, 0.00036525726318359375, 2.384185791015625e-05, 3.123283386230469e-05, 9.679794311523438e-05, 0.0005786418914794922, 6.532669067382812e-05, 4.458427429199219e-05, 0.0004553794860839844, 6.747245788574219e-05, 6.4849853515625e-05, 5.936622619628906e-05, 3.147125244140625e-05, 0.0005800724029541016, 8.320808410644531e-05, 3.218650817871094e-05, 3.0994415283203125e-05, 3.504753112792969e-05, 5.459785461425781e-05, 0.00029540061950683594, 4.673004150390625e-05, 4.839897155761719e-05, 2.9802322387695312e-05, 0.0002448558807373047, 4.3392181396484375e-05, 0.0003838539123535156, 3.600120544433594e-05, 3.0279159545898438e-05, 9.918212890625e-05, 0.0002713203430175781, 7.033348083496094e-05, 2.4318695068359375e-05, 0.0001499652862548828, 9.012222290039062e-05, 0.0001728534698486328, 0.000415802001953125, 0.0003478527069091797, 8.416175842285156e-05, 4.291534423828125e-05, 0.00030994415283203125, 7.510185241699219e-05, 0.0006644725799560547, 0.0004055500030517578, 3.314018249511719e-05, 6.604194641113281e-05, 0.0001239776611328125, 3.62396240234375e-05, 7.867813110351562e-05, 5.054473876953125e-05, 6.651878356933594e-05, 0.0005879402160644531, 0.00038242340087890625, 3.457069396972656e-05, 8.511543273925781e-05, 9.870529174804688e-05, 0.00033283233642578125, 0.0006501674652099609, 0.00019431114196777344, 4.482269287109375e-05, 0.0001010894775390625, 5.0067901611328125e-05, 8.106231689453125e-05, 0.0006208419799804688, 5.650520324707031e-05, 9.202957153320312e-05, 8.702278137207031e-05, 8.96453857421875e-05, 0.0004115104675292969, 0.0001938343048095703, 6.246566772460938e-05, 7.82012939453125e-05, 0.0006406307220458984, 8.726119995117188e-05, 0.0005445480346679688, 0.0006532669067382812, 0.00010848045349121094, 4.291534423828125e-05, 9.226799011230469e-05, 4.482269287109375e-05, 5.8650970458984375e-05, 3.552436828613281e-05, 5.698204040527344e-05, 8.797645568847656e-05, 6.318092346191406e-05, 3.409385681152344e-05, 3.933906555175781e-05, 4.744529724121094e-05, 0.0003178119659423828, 0.0003757476806640625, 3.790855407714844e-05, 6.437301635742188e-05, 4.506111145019531e-05, 7.963180541992188e-05, 6.437301635742188e-05, 5.054473876953125e-05, 4.5299530029296875e-05, 8.130073547363281e-05, 8.106231689453125e-05, 2.2411346435546875e-05, 0.0005395412445068359, 0.0002932548522949219, 0.00010943412780761719, 0.0002760887145996094, 0.0003352165222167969, 6.365776062011719e-05, 0.00029468536376953125, 6.389617919921875e-05, 0.00022149085998535156, 0.0001933574676513672, 3.2901763916015625e-05, 6.389617919921875e-05, 2.574920654296875e-05, 0.0002999305725097656, 0.00021338462829589844, 6.771087646484375e-05, 0.0002918243408203125, 8.58306884765625e-05, 5.555152893066406e-05, 3.62396240234375e-05, 5.173683166503906e-05, 8.320808410644531e-05, 3.9577484130859375e-05, 0.00021648406982421875, 0.0006432533264160156, 0.00010204315185546875, 6.29425048828125e-05, 7.033348083496094e-05, 9.202957153320312e-05, 8.296966552734375e-05, 0.000110626220703125, 6.628036499023438e-05, 0.00014328956604003906, 0.0001125335693359375, 8.249282836914062e-05, 9.775161743164062e-05, 9.083747863769531e-05, 6.651878356933594e-05, 9.751319885253906e-05, 6.914138793945312e-05, 0.0001201629638671875, 0.00010180473327636719, 9.560585021972656e-05, 9.441375732421875e-05, 8.416175842285156e-05, 8.845329284667969e-05, 7.2479248046875e-05, 0.00013184547424316406, 9.059906005859375e-05, 0.00010967254638671875, 6.532669067382812e-05, 8.296966552734375e-05, 7.43865966796875e-05, 5.316734313964844e-05, 4.57763671875e-05, 0.00011706352233886719, 0.00013375282287597656, 9.393692016601562e-05, 6.914138793945312e-05, 9.250640869140625e-05, 4.673004150390625e-05, 8.320808410644531e-05, 2.574920654296875e-05, 8.559226989746094e-05, 5.888938903808594e-05, 0.00010919570922851562, 2.765655517578125e-05, 3.170967102050781e-05, 6.651878356933594e-05, 3.814697265625e-05, 0.00011444091796875, 7.867813110351562e-05, 0.00012731552124023438, 7.891654968261719e-05, 5.316734313964844e-05, 0.0001289844512939453, 3.838539123535156e-05, 6.723403930664062e-05, 6.532669067382812e-05, 6.270408630371094e-05, 4.863739013671875e-05, 8.702278137207031e-05, 0.00010180473327636719, 6.723403930664062e-05, 6.318092346191406e-05, 9.703636169433594e-05, 5.8650970458984375e-05, 0.000118255615234375, 3.123283386230469e-05, 9.560585021972656e-05, 3.4809112548828125e-05, 4.8160552978515625e-05, 3.457069396972656e-05, 4.458427429199219e-05, 5.459785461425781e-05, 6.270408630371094e-05, 7.796287536621094e-05, 3.409385681152344e-05, 4.9114227294921875e-05, 5.888938903808594e-05, 2.8371810913085938e-05, 5.7220458984375e-05, 9.894371032714844e-05, 7.271766662597656e-05, 0.00010776519775390625, 3.8623809814453125e-05, 2.8848648071289062e-05, 7.033348083496094e-05, 6.866455078125e-05, 8.463859558105469e-05, 0.00011491775512695312, 0.00010395050048828125, 8.535385131835938e-05, 3.3855438232421875e-05, 6.67572021484375e-05, 0.00010395050048828125, 9.942054748535156e-05, 8.893013000488281e-05, 9.489059448242188e-05, 4.220008850097656e-05, 4.5299530029296875e-05, 8.440017700195312e-05, 6.4849853515625e-05, 7.963180541992188e-05, 0.00010943412780761719, 3.647804260253906e-05, 0.00011301040649414062, 0.00010538101196289062, 6.175041198730469e-05, 0.00013065338134765625, 7.510185241699219e-05, 7.510185241699219e-05, 7.152557373046875e-05, 2.5987625122070312e-05, 5.173683166503906e-05, 6.246566772460938e-05, 7.748603820800781e-05, 3.790855407714844e-05, 8.130073547363281e-05, 8.559226989746094e-05, 4.649162292480469e-05, 6.031990051269531e-05, 5.555152893066406e-05, 0.00010824203491210938, 6.699562072753906e-05, 6.4849853515625e-05, 3.600120544433594e-05, 9.918212890625e-05, 0.00011491775512695312, 9.226799011230469e-05, 5.888938903808594e-05, 5.340576171875e-05, 0.00010442733764648438, 3.504753112792969e-05, 8.463859558105469e-05, 6.008148193359375e-05, 6.246566772460938e-05, 5.459785461425781e-05, 5.459785461425781e-05, 7.724761962890625e-05, 0.00012445449829101562, 0.000102996826171875, 0.00012874603271484375, 4.5299530029296875e-05, 9.608268737792969e-05, 5.8650970458984375e-05, 9.226799011230469e-05, 3.814697265625e-05, 8.249282836914062e-05, 7.152557373046875e-05, 6.270408630371094e-05, 8.678436279296875e-05, 4.291534423828125e-05, 2.5510787963867188e-05, 7.414817810058594e-05, 8.606910705566406e-05, 0.00010275840759277344, 5.888938903808594e-05, 0.00010251998901367188, 5.269050598144531e-05, 3.361701965332031e-05, 5.316734313964844e-05, 0.00012040138244628906, 0.00012302398681640625, 4.124641418457031e-05, 9.465217590332031e-05, 6.556510925292969e-05, 9.393692016601562e-05, 8.726119995117188e-05, 7.367134094238281e-05, 5.269050598144531e-05, 8.535385131835938e-05, 6.198883056640625e-05, 2.8848648071289062e-05, 6.341934204101562e-05, 6.556510925292969e-05, 0.00012683868408203125, 9.417533874511719e-05, 3.337860107421875e-05, 7.2479248046875e-05, 5.412101745605469e-05, 0.0001125335693359375, 8.440017700195312e-05, 6.246566772460938e-05, 6.103515625e-05, 7.367134094238281e-05, 0.00011873245239257812, 7.796287536621094e-05, 8.821487426757812e-05, 7.081031799316406e-05, 7.486343383789062e-05, 0.00011086463928222656, 4.76837158203125e-05, 6.413459777832031e-05, 7.414817810058594e-05, 5.435943603515625e-05, 8.368492126464844e-05, 0.0001251697540283203, 5.650520324707031e-05, 5.125999450683594e-05, 5.7697296142578125e-05, 0.00010037422180175781, 8.368492126464844e-05, 5.030632019042969e-05, 9.775161743164062e-05, 6.270408630371094e-05, 0.0002307891845703125, 7.462501525878906e-05, 4.029273986816406e-05, 6.890296936035156e-05, 5.459785461425781e-05, 4.2438507080078125e-05, 4.1484832763671875e-05, 0.00010204315185546875, 5.221366882324219e-05, 8.177757263183594e-05, 9.34600830078125e-05, 5.0067901611328125e-05, 9.131431579589844e-05, 6.008148193359375e-05, 3.7670135498046875e-05, 0.00010848045349121094, 4.649162292480469e-05, 8.869171142578125e-05, 8.296966552734375e-05, 7.390975952148438e-05, 5.14984130859375e-05, 6.532669067382812e-05, 8.726119995117188e-05, 3.981590270996094e-05, 5.269050598144531e-05, 5.340576171875e-05, 4.291534423828125e-05, 7.677078247070312e-05, 7.510185241699219e-05, 8.845329284667969e-05, 6.437301635742188e-05, 6.4849853515625e-05, 6.270408630371094e-05, 8.130073547363281e-05, 5.698204040527344e-05, 9.989738464355469e-05, 6.175041198730469e-05, 9.322166442871094e-05, 4.601478576660156e-05, 0.00011014938354492188, 6.246566772460938e-05, 5.173683166503906e-05, 3.6716461181640625e-05, 9.202957153320312e-05, 5.745887756347656e-05, 8.749961853027344e-05, 2.8133392333984375e-05, 2.9087066650390625e-05, 0.0005698204040527344, 0.0007894039154052734, 6.318092346191406e-05, 0.0005564689636230469, 0.0007886886596679688, 0.0010526180267333984, 0.00016927719116210938, 4.029273986816406e-05, 7.033348083496094e-05, 7.700920104980469e-05, 0.0009493827819824219, 0.0003821849822998047, 0.0005476474761962891, 6.365776062011719e-05, 0.0001392364501953125, 0.00034046173095703125, 0.00017952919006347656, 0.00011157989501953125, 0.00048422813415527344, 8.559226989746094e-05, 0.0003185272216796875, 0.0005841255187988281, 0.0002548694610595703, 0.0002067089080810547, 0.0003898143768310547, 7.987022399902344e-05, 0.0006718635559082031, 0.0001704692840576172, 0.00012087821960449219, 0.0002663135528564453, 0.000125885009765625, 0.0007572174072265625, 0.0006186962127685547, 0.00010704994201660156, 0.00011658668518066406, 0.0007486343383789062, 0.00039124488830566406, 0.0010805130004882812, 0.0006163120269775391, 0.00016355514526367188, 0.0005776882171630859, 0.0006799697875976562, 0.0001232624053955078, 0.0003407001495361328, 0.0001289844512939453, 0.00010132789611816406, 0.00034427642822265625, 0.00047278404235839844, 0.0007863044738769531, 0.0001423358917236328, 0.0005981922149658203, 5.936622619628906e-05, 0.00018739700317382812, 5.8650970458984375e-05, 0.00023508071899414062, 0.0003643035888671875, 0.00015306472778320312, 0.0001766681671142578, 7.82012939453125e-05, 0.0005924701690673828, 3.0517578125e-05, 0.00017404556274414062, 9.441375732421875e-05, 4.8160552978515625e-05, 0.0009093284606933594, 5.054473876953125e-05, 0.00011992454528808594, 5.53131103515625e-05, 3.3855438232421875e-05, 0.0002455711364746094, 0.00011944770812988281, 0.0001518726348876953, 0.000400543212890625, 0.00014925003051757812, 0.00034809112548828125, 7.295608520507812e-05, 0.0007059574127197266, 0.0005147457122802734, 0.0003170967102050781, 0.00012969970703125, 0.0009181499481201172, 6.532669067382812e-05, 0.0003063678741455078, 0.0004286766052246094, 0.0001373291015625, 0.0004031658172607422, 0.00028228759765625, 0.0005462169647216797, 0.0004088878631591797, 0.00018644332885742188, 0.0002589225769042969, 0.0009055137634277344, 0.00024271011352539062, 8.392333984375e-05, 0.000118255615234375, 0.00039458274841308594, 0.0004775524139404297, 0.0007495880126953125, 0.0005316734313964844, 0.00026154518127441406, 3.8623809814453125e-05, 0.00018262863159179688, 0.00028634071350097656, 0.0006623268127441406, 0.0010154247283935547, 6.127357482910156e-05, 0.0003581047058105469, 0.0001838207244873047, 0.00019359588623046875, 0.00017261505126953125, 0.00043511390686035156, 0.00012230873107910156, 9.322166442871094e-05, 0.00019073486328125, 0.0005338191986083984, 0.0008177757263183594, 0.0004820823669433594, 0.00012421607971191406, 0.0003502368927001953, 0.0009372234344482422, 0.0004544258117675781, 0.00011754035949707031, 0.0002090930938720703, 0.0006024837493896484, 0.00015020370483398438, 0.0011234283447265625, 0.0008211135864257812, 0.00055694580078125, 0.0004677772521972656, 0.00013136863708496094, 7.104873657226562e-05, 0.00024890899658203125, 0.00013566017150878906, 8.821487426757812e-05, 0.0009713172912597656, 7.009506225585938e-05, 0.00017762184143066406, 0.00019049644470214844, 0.00011038780212402344, 0.0007152557373046875, 5.8650970458984375e-05, 8.20159912109375e-05, 0.00011897087097167969, 0.00026488304138183594, 8.249282836914062e-05, 0.000400543212890625, 0.0002791881561279297, 7.724761962890625e-05, 0.0005433559417724609, 0.0005774497985839844, 0.00042557716369628906, 0.00018906593322753906, 0.0007402896881103516, 0.0003771781921386719, 0.0005519390106201172, 7.271766662597656e-05, 0.0008463859558105469, 0.0001881122589111328, 0.0006372928619384766, 4.887580871582031e-05, 0.0001583099365234375, 0.00024962425231933594, 0.00044798851013183594, 0.0003001689910888672, 0.0010690689086914062, 6.508827209472656e-05, 0.0005755424499511719, 0.0009539127349853516, 0.0005681514739990234, 0.0005235671997070312, 9.751319885253906e-05, 0.0001342296600341797, 5.817413330078125e-05, 0.00023889541625976562, 6.365776062011719e-05, 0.00031447410583496094, 0.0005495548248291016, 9.822845458984375e-05, 0.0006151199340820312, 7.271766662597656e-05, 0.0001499652862548828, 0.0003628730773925781, 6.175041198730469e-05, 0.00017571449279785156, 0.0006511211395263672, 0.0001251697540283203, 5.793571472167969e-05, 0.0005362033843994141, 0.0007145404815673828, 0.0002415180206298828, 0.00010967254638671875, 0.0004036426544189453, 7.748603820800781e-05, 0.0008172988891601562, 0.0002655982971191406, 0.001012563705444336, 0.00011444091796875, 0.0007853507995605469, 0.00021123886108398438, 0.0002472400665283203, 0.0008177757263183594, 0.0007033348083496094, 6.604194641113281e-05, 0.00013709068298339844, 3.3855438232421875e-05, 0.0009074211120605469, 0.0002617835998535156, 0.000301361083984375, 0.0008082389831542969, 0.00036835670471191406, 0.0001678466796875, 0.00010800361633300781, 0.00012636184692382812, 0.00086212158203125, 5.221366882324219e-05, 8.106231689453125e-05, 0.0005805492401123047, 0.00037169456481933594, 0.00010895729064941406, 0.0006396770477294922, 0.0008053779602050781, 0.0001785755157470703, 0.0003781318664550781, 0.0005340576171875, 7.390975952148438e-05, 7.104873657226562e-05, 8.296966552734375e-05, 5.555152893066406e-05, 0.00022840499877929688, 7.152557373046875e-05, 0.00015997886657714844, 0.00012183189392089844, 0.00015592575073242188, 0.0005860328674316406, 0.00011920928955078125, 0.0007319450378417969, 0.0003616809844970703, 5.555152893066406e-05, 5.0067901611328125e-05, 0.0008025169372558594, 5.841255187988281e-05, 0.0006794929504394531, 0.0005826950073242188, 0.0003387928009033203, 0.0008146762847900391, 0.0004634857177734375, 0.00042176246643066406, 0.0007839202880859375, 0.0005886554718017578, 0.00025844573974609375, 0.002248048782348633, 0.00033211708068847656, 0.0002968311309814453, 0.00015091896057128906, 0.0006196498870849609, 0.0003371238708496094, 0.0003333091735839844, 0.00032973289489746094, 0.00011372566223144531, 0.0001506805419921875, 0.0001270771026611328, 0.0003612041473388672, 0.0001304149627685547, 6.699562072753906e-05, 0.00010967254638671875, 0.00014019012451171875, 0.00010228157043457031, 0.00012087821960449219, 0.0002186298370361328, 0.0007729530334472656, 0.00010967254638671875, 0.00016999244689941406, 0.0007228851318359375, 0.00025773048400878906, 0.00020694732666015625, 0.000209808349609375, 0.0007956027984619141, 0.0004525184631347656, 0.00024962425231933594, 0.00022363662719726562, 0.00021719932556152344, 0.00039887428283691406, 0.00018095970153808594, 0.0003135204315185547, 0.0005013942718505859, 0.0004718303680419922, 0.00020265579223632812, 0.0008893013000488281, 0.000362396240234375, 6.222724914550781e-05, 0.000827789306640625, 6.29425048828125e-05, 0.0002684593200683594, 0.00023031234741210938, 0.0013267993927001953, 0.00011110305786132812, 0.0006191730499267578, 0.0008573532104492188, 0.00018715858459472656, 0.00031447410583496094, 6.246566772460938e-05, 5.2928924560546875e-05, 0.00029540061950683594, 0.00023603439331054688, 0.0002732276916503906, 0.00016951560974121094, 0.0006504058837890625, 0.00037407875061035156, 0.00043487548828125, 7.486343383789062e-05, 0.00010919570922851562, 0.0006489753723144531, 0.0007905960083007812, 7.987022399902344e-05, 0.00024318695068359375, 7.462501525878906e-05, 0.0001933574676513672, 0.0007212162017822266, 4.839897155761719e-05, 0.0011320114135742188, 0.0012111663818359375, 0.0002772808074951172, 0.00015044212341308594, 7.867813110351562e-05, 0.00020551681518554688, 0.0005049705505371094, 6.103515625e-05, 0.0002751350402832031, 0.0004448890686035156, 0.0003190040588378906, 0.00010919570922851562, 0.0005300045013427734, 0.0006411075592041016, 0.0008487701416015625, 0.00022411346435546875, 0.0003848075866699219, 0.00035381317138671875, 0.0006215572357177734, 0.0005533695220947266, 0.0007147789001464844, 0.000911712646484375, 0.00010657310485839844, 0.0002791881561279297, 7.128715515136719e-05, 0.00016450881958007812, 6.270408630371094e-05, 0.00011301040649414062, 0.0003399848937988281, 9.799003601074219e-05, 0.0012347698211669922, 0.0002582073211669922, 6.961822509765625e-05, 0.00041985511779785156, 0.0002570152282714844, 0.0004138946533203125, 0.00021457672119140625, 4.172325134277344e-05, 0.0005118846893310547, 0.0003349781036376953, 4.124641418457031e-05, 0.0006480216979980469, 0.0004892349243164062, 0.00013947486877441406, 0.0002865791320800781, 0.00038909912109375, 0.0002028942108154297, 7.724761962890625e-05, 0.0002758502960205078, 0.00010323524475097656, 0.0002665519714355469, 0.0006563663482666016, 0.0003077983856201172, 0.00021696090698242188, 0.0003399848937988281, 7.104873657226562e-05, 0.0005259513854980469, 0.00028634071350097656, 7.414817810058594e-05, 0.0006177425384521484, 0.00015163421630859375, 0.00016641616821289062, 0.00020384788513183594, 0.000171661376953125, 0.00024700164794921875, 0.0007262229919433594, 0.0002529621124267578, 6.890296936035156e-05, 6.985664367675781e-05, 7.367134094238281e-05, 0.00011849403381347656, 7.367134094238281e-05, 0.00022363662719726562, 0.0003523826599121094, 0.00015926361083984375, 0.0006625652313232422, 0.0006434917449951172, 0.0008788108825683594, 0.0002129077911376953, 0.00011873245239257812, 0.0009064674377441406, 0.0002524852752685547, 0.00010776519775390625, 9.274482727050781e-05, 0.0006432533264160156, 0.0001327991485595703, 0.0005815029144287109, 8.988380432128906e-05, 0.0006611347198486328, 0.0007460117340087891, 0.0004911422729492188, 0.00014352798461914062, 0.00017952919006347656, 5.364418029785156e-05, 0.00019741058349609375, 0.0005013942718505859, 0.0012116432189941406, 0.0004878044128417969, 0.00013566017150878906, 0.0003457069396972656, 0.0006163120269775391, 0.00010395050048828125, 0.0007030963897705078, 0.0007386207580566406, 0.0001423358917236328, 5.888938903808594e-05, 0.0001773834228515625, 0.0002663135528564453, 0.00023746490478515625, 6.413459777832031e-05, 3.695487976074219e-05, 9.036064147949219e-05, 0.0003273487091064453, 0.0015552043914794922, 9.202957153320312e-05, 7.987022399902344e-05, 0.00015974044799804688, 0.00010061264038085938, 0.000461578369140625, 0.00018525123596191406, 0.00018668174743652344, 0.0007739067077636719, 5.7220458984375e-05, 6.175041198730469e-05, 0.00010037422180175781, 5.936622619628906e-05, 0.00010800361633300781, 0.00041937828063964844, 0.00010323524475097656, 0.0002593994140625, 0.0006237030029296875, 0.00014138221740722656, 0.00025463104248046875, 0.00015091896057128906, 5.0067901611328125e-05, 0.00034546852111816406, 3.409385681152344e-05, 0.0004496574401855469, 0.0004265308380126953, 0.00020241737365722656, 9.393692016601562e-05, 0.0002827644348144531, 0.000335693359375, 0.0004374980926513672, 6.0558319091796875e-05, 0.00015425682067871094, 0.00037026405334472656, 0.000164031982421875, 9.870529174804688e-05, 0.00010561943054199219, 0.00019550323486328125, 0.00010704994201660156, 0.00014400482177734375, 0.00011920928955078125, 0.00013875961303710938, 0.00011706352233886719, 0.0009183883666992188, 0.0004985332489013672, 0.00023031234741210938, 3.8623809814453125e-05, 0.0002295970916748047, 4.458427429199219e-05, 0.0001506805419921875, 0.0010676383972167969, 0.0007152557373046875, 0.00015282630920410156, 9.989738464355469e-05, 0.00010776519775390625, 0.0002453327178955078, 0.00018525123596191406, 0.00017714500427246094, 6.222724914550781e-05, 0.00023102760314941406, 0.00010347366333007812, 0.00045013427734375, 0.0004220008850097656, 0.0003979206085205078, 0.00012111663818359375, 0.0008690357208251953, 0.0008847713470458984, 0.00026679039001464844, 0.005445241928100586, 0.00028777122497558594, 0.0010082721710205078, 0.0007271766662597656, 0.0005292892456054688, 0.0001766681671142578, 0.0005977153778076172, 0.0005791187286376953, 8.058547973632812e-05, 0.0005526542663574219, 0.00021600723266601562, 0.0001595020294189453, 0.0003170967102050781, 0.0005645751953125, 0.00014638900756835938, 0.0003101825714111328, 0.0003590583801269531, 7.62939453125e-05, 0.0002231597900390625, 0.0004630088806152344, 0.0005869865417480469, 0.0006239414215087891, 0.00021028518676757812, 0.00019478797912597656, 0.0013415813446044922, 0.0002493858337402344, 0.00017309188842773438, 0.0003211498260498047, 0.00024628639221191406, 0.0003895759582519531, 0.0007925033569335938, 0.00024056434631347656, 0.0008919239044189453, 0.0002288818359375, 0.0001723766326904297, 0.00013566017150878906, 0.0004925727844238281, 0.0001723766326904297, 6.103515625e-05, 0.0002732276916503906, 0.00030350685119628906, 0.0002086162567138672, 0.0006785392761230469, 0.0002582073211669922, 0.000804901123046875, 0.0007526874542236328, 0.00012636184692382812, 7.224082946777344e-05, 0.0008041858673095703, 0.0007889270782470703, 0.0001251697540283203, 6.008148193359375e-05, 0.00012159347534179688, 0.0006275177001953125, 0.00011897087097167969, 5.364418029785156e-05, 0.00011134147644042969, 0.0008068084716796875, 0.0004980564117431641, 0.00017404556274414062, 0.0002307891845703125, 0.00019359588623046875, 5.91278076171875e-05, 0.0002663135528564453, 0.0002779960632324219, 0.0007402896881103516, 0.00014781951904296875, 0.0009341239929199219, 0.000164031982421875, 0.00093841552734375, 0.00019979476928710938, 0.0001304149627685547, 0.0006964206695556641, 0.00020432472229003906, 0.0003292560577392578, 0.00016164779663085938, 0.00011277198791503906, 0.0011186599731445312, 0.0002143383026123047, 0.00041413307189941406, 0.00023603439331054688, 9.584426879882812e-05, 0.0005142688751220703, 5.5789947509765625e-05, 0.0006372928619384766, 0.00011157989501953125, 0.0001270771026611328, 0.0010385513305664062, 5.364418029785156e-05, 0.0006546974182128906, 0.0002186298370361328, 0.0006961822509765625, 5.698204040527344e-05, 9.131431579589844e-05, 0.0004754066467285156, 0.00025534629821777344, 7.581710815429688e-05, 0.0012121200561523438, 0.0006098747253417969, 0.0007047653198242188, 0.00022172927856445312, 0.0010001659393310547, 0.00010776519775390625, 0.00012063980102539062, 0.00039458274841308594, 7.128715515136719e-05, 0.00016427040100097656, 0.0004870891571044922, 9.226799011230469e-05, 0.00010848045349121094, 7.200241088867188e-05, 0.0010051727294921875, 0.001631021499633789, 0.0007328987121582031, 0.0003497600555419922, 0.0011172294616699219, 0.0001647472381591797, 6.365776062011719e-05, 0.0004482269287109375, 5.4836273193359375e-05, 0.00010156631469726562, 0.0006237030029296875, 0.00046563148498535156, 0.001264810562133789, 0.0004508495330810547, 0.00015878677368164062, 0.0007460117340087891, 0.00012564659118652344, 0.00018644332885742188, 0.00012493133544921875, 0.00010228157043457031, 0.0003199577331542969, 0.00011396408081054688, 0.00033664703369140625, 0.000598907470703125, 0.0006532669067382812, 0.0003006458282470703, 3.933906555175781e-05, 0.0004646778106689453, 0.0007441043853759766, 0.00022983551025390625, 0.0014815330505371094, 0.0010809898376464844, 7.772445678710938e-05, 0.0009443759918212891, 0.0001087188720703125, 6.604194641113281e-05, 0.0002548694610595703, 0.00012874603271484375, 9.083747863769531e-05, 0.0004944801330566406, 0.0004801750183105469, 0.00017261505126953125, 0.0010905265808105469, 0.0008759498596191406, 4.8160552978515625e-05, 0.0005598068237304688, 0.00012445449829101562, 0.00023746490478515625, 0.00017452239990234375, 0.0005049705505371094, 0.0007266998291015625, 0.0004057884216308594, 0.00036787986755371094, 0.00018525123596191406, 0.0002682209014892578, 0.00025916099548339844, 0.00022935867309570312, 0.00021147727966308594, 0.0006392002105712891, 0.0001728534698486328, 0.0002276897430419922, 0.00010371208190917969, 0.000244140625, 0.0008702278137207031, 0.0006372928619384766, 0.0003936290740966797, 0.00011038780212402344, 0.0002453327178955078, 0.000110626220703125, 0.00758814811706543, 0.00010991096496582031, 0.0005371570587158203, 0.00046706199645996094, 6.532669067382812e-05, 0.00042057037353515625, 0.0001494884490966797, 0.00016236305236816406, 9.751319885253906e-05, 0.00028634071350097656, 0.00037479400634765625, 0.0001811981201171875, 0.0001728534698486328, 0.0005311965942382812, 0.0002162456512451172, 0.00011944770812988281, 0.00017714500427246094, 0.00012993812561035156, 0.0006117820739746094, 0.0005211830139160156, 0.0006325244903564453, 0.00020885467529296875, 0.00016880035400390625, 0.000244140625, 0.0001628398895263672, 0.00041961669921875, 0.0008213520050048828, 0.00014472007751464844, 0.00037789344787597656, 0.0004096031188964844, 0.00020074844360351562, 9.226799011230469e-05, 9.870529174804688e-05, 0.00016164779663085938, 0.0003097057342529297, 0.00037670135498046875, 0.00010251998901367188, 0.0006425380706787109, 8.559226989746094e-05, 0.0011212825775146484, 0.00041556358337402344, 0.0006897449493408203, 0.0005276203155517578, 0.0003731250762939453, 9.5367431640625e-05, 6.389617919921875e-05, 0.00011610984802246094, 0.0001418590545654297, 0.0001544952392578125, 0.0008375644683837891, 0.00017833709716796875, 7.843971252441406e-05, 0.000377655029296875, 0.000408172607421875, 0.0001876354217529297, 0.0007340908050537109, 0.0003695487976074219, 0.0007653236389160156, 0.00011610984802246094, 0.00045800209045410156, 0.0004906654357910156, 0.00010585784912109375, 0.0004165172576904297, 0.000217437744140625, 0.0003478527069091797, 0.00019311904907226562, 0.0002110004425048828, 5.602836608886719e-05, 0.00014066696166992188, 9.870529174804688e-05, 0.00010466575622558594, 0.0009412765502929688, 0.00011420249938964844, 8.726119995117188e-05, 8.797645568847656e-05, 0.00011086463928222656, 0.0006039142608642578, 0.00010704994201660156, 0.0005409717559814453, 0.0009253025054931641, 0.00016570091247558594, 0.0003044605255126953, 0.0009565353393554688, 0.0005443096160888672, 0.00018072128295898438, 0.00033020973205566406, 9.799003601074219e-05, 0.0007271766662597656, 0.0007851123809814453, 0.000171661376953125, 0.0006558895111083984, 0.0005767345428466797, 0.0006749629974365234, 0.0002932548522949219, 0.00010776519775390625, 0.00046181678771972656, 0.0004150867462158203, 0.00022602081298828125, 0.00014638900756835938, 0.0002193450927734375, 0.00017976760864257812, 0.00010919570922851562, 6.175041198730469e-05, 0.0001437664031982422, 4.839897155761719e-05, 9.465217590332031e-05, 0.0004570484161376953, 0.00012159347534179688, 0.0005700588226318359, 8.535385131835938e-05, 0.00017786026000976562, 0.0005686283111572266, 0.00014328956604003906, 9.560585021972656e-05, 0.000461578369140625, 0.0004405975341796875, 0.0003845691680908203, 0.00020074844360351562, 0.00014781951904296875, 0.00012183189392089844, 0.00026535987854003906, 0.0003707408905029297, 5.173683166503906e-05, 0.00010609626770019531, 0.0004487037658691406, 0.0002052783966064453, 0.00014662742614746094, 0.00016999244689941406, 0.00015234947204589844, 0.0005929470062255859, 9.465217590332031e-05, 0.00013136863708496094, 0.00010514259338378906, 5.340576171875e-05, 7.510185241699219e-05, 0.00021696090698242188, 0.0007007122039794922, 0.00016236305236816406, 0.0001220703125, 0.00012135505676269531, 0.00023365020751953125, 0.0006780624389648438, 8.726119995117188e-05, 0.0005397796630859375, 0.00011777877807617188, 0.00011301040649414062, 7.462501525878906e-05, 0.0006897449493408203, 6.67572021484375e-05, 0.00037789344787597656, 0.00010991096496582031, 0.00043129920959472656, 8.630752563476562e-05, 0.0006921291351318359, 0.00029730796813964844, 0.0003364086151123047, 0.0010285377502441406, 8.440017700195312e-05, 8.58306884765625e-05, 0.00027561187744140625, 0.0007717609405517578, 0.0005846023559570312, 5.412101745605469e-05, 0.00013828277587890625, 5.888938903808594e-05, 0.00010371208190917969, 3.7670135498046875e-05, 9.202957153320312e-05, 9.632110595703125e-05, 0.0005657672882080078, 6.413459777832031e-05, 0.0009589195251464844, 0.0005011558532714844, 0.000362396240234375, 0.00048732757568359375, 0.00017833709716796875, 0.000865936279296875, 0.0006377696990966797, 0.0003714561462402344, 0.0001614093780517578, 0.0009124279022216797, 0.00023984909057617188, 0.0001785755157470703, 0.0005435943603515625, 0.0005736351013183594, 0.0009548664093017578, 7.843971252441406e-05, 0.0004894733428955078, 9.989738464355469e-05, 0.00042057037353515625, 0.0002162456512451172, 0.00011277198791503906, 0.0001513957977294922, 0.0006937980651855469, 0.0005004405975341797, 8.106231689453125e-05, 0.0005953311920166016, 0.0008282661437988281, 0.0001583099365234375, 0.0002808570861816406, 0.0008258819580078125, 3.8623809814453125e-05, 0.0006537437438964844, 0.0007312297821044922, 0.000537872314453125, 0.0011794567108154297, 0.001177072525024414, 6.723403930664062e-05, 0.000518798828125, 0.0005598068237304688, 6.842613220214844e-05, 0.00010347366333007812, 6.389617919921875e-05, 0.0001475811004638672, 3.6716461181640625e-05, 0.00034356117248535156, 0.00011515617370605469, 3.2901763916015625e-05, 0.0003285408020019531, 0.0001857280731201172, 0.0005395412445068359, 0.0001633167266845703, 0.0003647804260253906, 0.0006704330444335938, 0.0007419586181640625, 0.0003371238708496094, 0.0001766681671142578, 0.00014925003051757812, 0.0002624988555908203, 0.0002434253692626953, 6.794929504394531e-05, 0.0001289844512939453, 0.0004715919494628906, 0.0006237030029296875, 9.036064147949219e-05, 0.0004813671112060547, 0.0007035732269287109, 0.0009222030639648438, 0.0008578300476074219, 0.0006067752838134766, 0.000186920166015625, 0.00012135505676269531, 0.00017547607421875, 0.0009405612945556641, 0.0007107257843017578, 0.00110626220703125, 0.00017714500427246094, 0.0006225109100341797, 0.0005412101745605469, 4.506111145019531e-05, 0.00024318695068359375, 0.0005908012390136719, 0.00031280517578125, 9.918212890625e-05, 0.0006046295166015625, 0.0004975795745849609, 0.0001499652862548828, 0.0005688667297363281, 0.00013971328735351562, 0.00017762184143066406, 0.00013065338134765625, 9.942054748535156e-05, 0.00015664100646972656, 0.0006072521209716797, 0.0005156993865966797, 0.00029659271240234375, 0.0008325576782226562, 8.869171142578125e-05, 0.00044155120849609375, 0.00014352798461914062, 6.008148193359375e-05, 0.00011944770812988281, 0.0004191398620605469, 9.775161743164062e-05, 0.0004570484161376953, 0.0002453327178955078, 0.0009286403656005859, 0.00044608116149902344, 0.0008955001831054688, 5.602836608886719e-05, 0.00010466575622558594, 0.0005402565002441406, 0.00011849403381347656, 0.0010142326354980469, 0.0001842975616455078, 0.0001423358917236328, 5.2928924560546875e-05, 3.147125244140625e-05, 0.0007505416870117188, 0.00013971328735351562, 0.000244140625, 0.0006103515625, 0.00013399124145507812, 5.91278076171875e-05, 0.0006091594696044922, 0.00031375885009765625, 0.00016260147094726562, 0.00010251998901367188, 2.86102294921875e-05, 0.0007112026214599609, 0.0002300739288330078, 0.0007030963897705078, 0.0008273124694824219, 0.0005121231079101562, 0.00017380714416503906, 0.00018167495727539062, 0.0005590915679931641, 0.00011968612670898438, 8.797645568847656e-05, 0.0007498264312744141, 0.00028967857360839844, 0.0006444454193115234, 0.0001087188720703125, 0.00012946128845214844, 0.0001239776611328125, 0.0005154609680175781, 0.0001277923583984375, 0.0002422332763671875, 0.000499725341796875, 0.0005383491516113281, 0.0003998279571533203, 0.0009765625, 0.00025725364685058594, 0.00026345252990722656, 0.0006244182586669922, 0.0012340545654296875, 9.632110595703125e-05, 5.340576171875e-05, 8.392333984375e-05, 0.00034427642822265625, 0.00012493133544921875, 8.749961853027344e-05, 7.510185241699219e-05, 5.745887756347656e-05, 8.7738037109375e-05, 4.839897155761719e-05, 0.0001385211944580078, 0.0001513957977294922, 0.0004904270172119141, 0.0005550384521484375, 4.8160552978515625e-05, 0.00018930435180664062, 0.00011777877807617188, 8.749961853027344e-05, 0.00017118453979492188, 6.937980651855469e-05, 0.0004189014434814453, 2.8848648071289062e-05, 7.295608520507812e-05, 4.458427429199219e-05, 0.0006921291351318359, 0.0006604194641113281, 9.059906005859375e-05, 0.00017023086547851562, 5.841255187988281e-05, 0.0002541542053222656, 0.00043654441833496094, 0.0004520416259765625, 0.0006859302520751953, 0.00030350685119628906, 0.00011181831359863281, 8.7738037109375e-05, 0.0005311965942382812, 0.00012636184692382812, 0.0005979537963867188, 0.00017261505126953125, 5.745887756347656e-05, 0.00010466575622558594, 0.0003376007080078125, 4.315376281738281e-05, 0.0001914501190185547, 8.96453857421875e-05, 8.726119995117188e-05, 0.0005490779876708984, 0.0007708072662353516, 0.0007109642028808594, 5.7697296142578125e-05, 9.322166442871094e-05, 0.0005128383636474609, 0.0001595020294189453, 7.653236389160156e-05, 0.0005726814270019531, 0.000133514404296875, 4.792213439941406e-05, 6.556510925292969e-05, 5.984306335449219e-05, 0.0005605220794677734, 3.552436828613281e-05, 7.152557373046875e-05, 9.179115295410156e-05, 0.00018143653869628906, 0.00010991096496582031, 0.0003077983856201172, 0.00018262863159179688, 0.0005395412445068359, 3.647804260253906e-05, 0.00013327598571777344, 0.0004887580871582031, 0.00035572052001953125, 4.9114227294921875e-05, 0.0001690387725830078, 0.00047469139099121094, 0.00046539306640625, 4.172325134277344e-05, 0.0009391307830810547, 0.00012946128845214844, 0.00017404556274414062, 0.00029468536376953125, 0.00012612342834472656, 0.0005121231079101562, 0.0002415180206298828, 6.031990051269531e-05, 6.031990051269531e-05, 8.058547973632812e-05, 7.2479248046875e-05, 7.224082946777344e-05, 9.250640869140625e-05, 8.58306884765625e-05, 7.176399230957031e-05, 9.322166442871094e-05, 3.361701965332031e-05, 6.175041198730469e-05, 7.581710815429688e-05, 4.100799560546875e-05, 9.107589721679688e-05, 5.8650970458984375e-05, 7.367134094238281e-05, 6.318092346191406e-05, 9.441375732421875e-05, 3.075599670410156e-05, 9.1552734375e-05, 5.984306335449219e-05, 6.842613220214844e-05, 4.935264587402344e-05, 7.891654968261719e-05, 8.893013000488281e-05, 7.224082946777344e-05, 6.508827209472656e-05, 5.817413330078125e-05, 8.392333984375e-05, 3.2901763916015625e-05, 5.1021575927734375e-05, 7.510185241699219e-05, 5.555152893066406e-05, 6.175041198730469e-05, 6.365776062011719e-05, 3.981590270996094e-05, 3.2901763916015625e-05, 6.747245788574219e-05, 9.179115295410156e-05, 3.218650817871094e-05, 4.410743713378906e-05, 8.082389831542969e-05, 8.869171142578125e-05, 9.298324584960938e-05, 8.559226989746094e-05, 8.845329284667969e-05, 9.417533874511719e-05, 6.413459777832031e-05, 3.2901763916015625e-05, 5.0067901611328125e-05, 8.845329284667969e-05, 7.724761962890625e-05, 4.1961669921875e-05, 7.486343383789062e-05, 5.316734313964844e-05, 7.939338684082031e-05, 9.5367431640625e-05, 6.0558319091796875e-05, 5.125999450683594e-05, 4.744529724121094e-05, 9.989738464355469e-05, 4.7206878662109375e-05, 5.555152893066406e-05, 5.269050598144531e-05, 5.4836273193359375e-05, 7.05718994140625e-05, 4.601478576660156e-05, 4.935264587402344e-05, 8.440017700195312e-05, 8.463859558105469e-05, 4.7206878662109375e-05, 6.151199340820312e-05, 0.00010824203491210938, 7.557868957519531e-05, 7.915496826171875e-05, 6.079673767089844e-05, 6.914138793945312e-05, 3.0517578125e-05, 5.602836608886719e-05, 4.696846008300781e-05, 4.482269287109375e-05, 5.698204040527344e-05, 5.3882598876953125e-05, 3.62396240234375e-05, 7.843971252441406e-05, 7.748603820800781e-05, 9.036064147949219e-05, 4.553794860839844e-05, 4.76837158203125e-05, 9.34600830078125e-05, 6.318092346191406e-05, 7.939338684082031e-05, 6.699562072753906e-05, 4.7206878662109375e-05, 7.534027099609375e-05, 4.887580871582031e-05, 5.173683166503906e-05, 7.176399230957031e-05, 3.2901763916015625e-05, 5.507469177246094e-05, 7.557868957519531e-05, 9.036064147949219e-05, 9.489059448242188e-05, 9.202957153320312e-05, 3.4809112548828125e-05, 6.937980651855469e-05, 3.24249267578125e-05, 4.792213439941406e-05, 5.555152893066406e-05, 5.412101745605469e-05, 0.0001068115234375, 4.839897155761719e-05, 8.96453857421875e-05, 6.341934204101562e-05, 4.601478576660156e-05, 5.364418029785156e-05, 6.175041198730469e-05, 8.630752563476562e-05, 5.626678466796875e-05, 9.751319885253906e-05, 9.489059448242188e-05, 5.459785461425781e-05, 5.555152893066406e-05, 7.295608520507812e-05, 6.270408630371094e-05, 5.650520324707031e-05, 3.409385681152344e-05, 3.3855438232421875e-05, 4.601478576660156e-05, 2.384185791015625e-05, 7.534027099609375e-05, 3.24249267578125e-05, 4.887580871582031e-05, 8.845329284667969e-05, 2.288818359375e-05, 8.177757263183594e-05, 5.245208740234375e-05, 6.127357482910156e-05, 6.532669067382812e-05, 9.846687316894531e-05, 3.314018249511719e-05, 4.76837158203125e-05, 9.369850158691406e-05, 5.030632019042969e-05, 7.867813110351562e-05, 7.581710815429688e-05, 4.649162292480469e-05, 2.4557113647460938e-05, 7.152557373046875e-05, 6.937980651855469e-05, 7.319450378417969e-05, 6.175041198730469e-05, 3.409385681152344e-05, 5.054473876953125e-05, 3.6716461181640625e-05, 3.24249267578125e-05, 8.7738037109375e-05, 7.534027099609375e-05, 4.124641418457031e-05, 7.534027099609375e-05, 7.081031799316406e-05, 4.553794860839844e-05, 8.273124694824219e-05, 8.440017700195312e-05, 4.5299530029296875e-05, 0.00010371208190917969, 4.673004150390625e-05, 8.58306884765625e-05, 7.939338684082031e-05, 4.863739013671875e-05, 5.078315734863281e-05, 4.506111145019531e-05, 9.560585021972656e-05, 5.793571472167969e-05, 7.939338684082031e-05, 0.00015354156494140625, 7.939338684082031e-05, 3.886222839355469e-05, 3.910064697265625e-05, 8.869171142578125e-05, 8.296966552734375e-05, 7.987022399902344e-05, 5.793571472167969e-05, 7.772445678710938e-05, 5.8650970458984375e-05, 4.363059997558594e-05, 4.0531158447265625e-05, 3.719329833984375e-05, 5.626678466796875e-05, 3.3855438232421875e-05, 4.57763671875e-05, 6.604194641113281e-05, 5.8650970458984375e-05, 7.390975952148438e-05, 4.029273986816406e-05, 6.914138793945312e-05, 4.839897155761719e-05, 7.843971252441406e-05, 0.00012946128845214844, 9.72747802734375e-05, 0.00010418891906738281, 6.67572021484375e-05, 0.00017976760864257812, 0.00013947486877441406, 8.678436279296875e-05, 0.00011873245239257812, 0.00011730194091796875, 8.988380432128906e-05, 9.751319885253906e-05, 6.723403930664062e-05, 4.76837158203125e-05, 0.0001251697540283203, 0.00014734268188476562, 8.320808410644531e-05, 9.703636169433594e-05, 8.678436279296875e-05, 9.417533874511719e-05, 0.00011897087097167969, 5.626678466796875e-05, 7.581710815429688e-05, 4.267692565917969e-05, 5.936622619628906e-05, 9.441375732421875e-05, 9.059906005859375e-05, 5.316734313964844e-05, 5.1975250244140625e-05, 6.532669067382812e-05, 8.58306884765625e-05, 9.584426879882812e-05, 8.034706115722656e-05, 7.939338684082031e-05, 6.365776062011719e-05, 8.869171142578125e-05, 3.910064697265625e-05, 5.817413330078125e-05, 4.649162292480469e-05, 2.5272369384765625e-05, 6.175041198730469e-05, 8.916854858398438e-05, 7.700920104980469e-05, 4.9591064453125e-05, 3.981590270996094e-05], 'rps': 2321.436233516264}\n", "BENCHMARK REDIS\n", "{'count': 2250, 'average_latency': np.float64(3.755495283338759e-05), 'p50': np.float64(3.504753112792969e-05), 'p90': np.float64(4.2438507080078125e-05), 'p95': np.float64(4.601478576660156e-05), 'p99': np.float64(6.820678710937455e-05), 'std': np.float64(1.239698140422363e-05), 'min': np.float64(3.266334533691406e-05), 'max': np.float64(0.0005145072937011719), 'total_time': 0.11128711700439453, 'latency': array([5.14507294e-04, 9.51290131e-05, 5.05447388e-05, ...,\n", " 3.45706940e-05, 3.50475311e-05, 3.71932983e-05], shape=(2250,)), 'rps': 20217.973657374478}\n", "MEMORY USAGE REDIS\n", "Memory: 9.48M (9939256 bytes)\n", " test statistic p-value \\\n", "0 paired t-test 18.849559 4.156724e-73 \n", "1 KS-test 0.869154 0.000000e+00 \n", "\n", " interpretation \n", "0 H0: mean latency equal (Redis vs Syzygy) \n", "1 H0: same latency distribution (Redis vs Syzygy) \n" ] } ], "source": [ "def statistical_tests(bench_redis, bench_syzygy):\n", " redis_lat = np.array(bench_redis[\"latency\"], dtype=np.float64)\n", " syz_lat = np.array(bench_syzygy[\"latency\"], dtype=np.float64)\n", " \n", " min_len = min(len(redis_lat), len(syz_lat))\n", " redis_lat = redis_lat[:min_len]\n", " syz_lat = syz_lat[:min_len]\n", "\n", " # PAIRED T-Test (Comparaison median)\n", " t_stat, t_p = stats.ttest_rel(syz_lat, redis_lat)\n", "\n", " # KS Test (Comparaison distribution)\n", " ks_stat, ks_p = stats.ks_2samp(syz_lat, redis_lat)\n", "\n", " df_stats = pd.DataFrame({\n", " \"test\": [\"paired t-test\", \"KS-test\"],\n", " \"statistic\": [t_stat, ks_stat],\n", " \"p-value\": [t_p, ks_p],\n", " \"interpretation\": [\n", " \"H0: mean latency equal (Redis vs Syzygy)\",\n", " \"H0: same latency distribution (Redis vs Syzygy)\"\n", " ]\n", " })\n", "\n", " print(df_stats)\n", " return df_stats\n", "\n", "\n", "def run_experiment():\n", " redis_server = redis.Redis(host=\"localhost\", port=6379, db=0)\n", " tablebases = chess.syzygy.open_tablebase(\"tablebases/\")\n", "\n", " configs = [\"KBNvK\",\"KPvK\",\"KPvKP\",\"KQvK\",\"KQvKP\",\"KRBvKP\",\"KRPvKP\",\"KRvK\",\"KRvKB\"]\n", "\n", " all_positions = []\n", " for config in configs:\n", " all_positions += get_250_random_position(tablebases, config)\n", "\n", " #Filter to have 2 paired test have the same N\n", " valid_positions = []\n", " for board in all_positions:\n", " try:\n", " tablebases.probe_wdl(board)\n", " valid_positions.append(board)\n", " except chess.syzygy.MissingTableError:\n", " continue\n", " \n", " # latency arrays have different length so but to boardcast we have to use compare same length\n", " # that why it's important to \n", " print(f\"Redis latency array length: {len(results['bench_redis']['latency'])}\")\n", " print(f\"Syzygy latency array length: {len(results['bench_syzygy']['latency'])}\")\n", "\n", " print(\"BENCHMARK SYZYGY\")\n", " bench_syzygy = benchmark_syzygy(tablebases, valid_positions)\n", " print(bench_syzygy)\n", "\n", " print(\"BENCHMARK REDIS\")\n", " bench_redis = benchmark_redis(redis_server, valid_positions)\n", " print(bench_redis)\n", "\n", " print(\"MEMORY USAGE REDIS\")\n", " mem_bytes, mem_human = get_redis_memory(redis_server)\n", " print(f\"Memory: {mem_human} ({mem_bytes} bytes)\")\n", "\n", " return {\n", " \"bench_syzygy\": bench_syzygy,\n", " \"bench_redis\": bench_redis\n", " }\n", "\n", "if __name__ == \"__main__\":\n", " results = run_experiment()\n", " statistical_tests(results[\"bench_redis\"], results[\"bench_syzygy\"])" ] }, { "cell_type": "code", "execution_count": null, "id": "a68060e1-0052-4446-a0d3-40bc1756f210", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }