파이썬, 셀레니움 XHR 데이터 확인 예제

<참조> https://pythondocs.net/selenium/%EC%85%80%EB%A0%88%EB%8B%88%EC%9B%80-%ED%81%AC%EB%A1%A4%EB%9F%AC-%EA%B8%B0%EB%B3%B8-%EC%82%AC%EC%9A%A9%EB%B2%95

파이썬, 셀레니움 XHR 데이터 확인 예제 

 

 

from selenium.webdriver import DesiredCapabilities

import json

 

capabilities = DesiredCapabilities.CHROME

capabilities["goog:loggingPrefs"] = {"performance": "ALL"}  # newer: goog:loggingPrefs

 

try:

    s = Service(f'./{chrome_ver}/chromedriver.exe')

    driver = webdriver.Chrome(service=s, options=option, desired_capabilities=capabilities)

except:

    chromedriver_autoinstaller.install(True)

    s = Service(f'./{chrome_ver}/chromedriver.exe')

    driver = webdriver.Chrome(service=s, options=option, desired_capabilities=capabilities)

   

driver.implicitly_wait(10)

driver.get('blablablabla~~~')

 

logs_raw = driver.get_log("performance")

logs = [json.loads(lr["message"])["message"] for lr in logs_raw]

 

def log_filter(log_):

    # is an actual response and json

    return ( log_["method"] == "Network.responseReceived"  and

                 "json" in log_["params"]["response"]["mimeType"] )

 

for log in filter(log_filter, logs):

    request_id = log["params"]["requestId"]

    resp_url = log["params"]["response"]["url"]

    print(f"Caught {resp_url}")

    print(driver.execute_cdp_cmd("Network.getResponseBody", {"requestId": request_id}))

 

 

[참고]  XHR (XML Http Request) 이란

https://iotengineer.tistory.com/3

XHR (XML Http Request) 이란