ByJJoon's Lab

한동안 크롤링 스크립트를 잘 운영하다 어느날부터 아래와 같은 에러가 발생.

  File "/home/pi/apti/lib/driver_open_close.py", line 23, in open_Driver
    driver = webdriver.Chrome(service=service, options=options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/.venv/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
    super().__init__(
  File "/home/pi/.venv/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 66, in __init__
    super().__init__(command_executor=executor, options=options)
  File "/home/pi/.venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 250, in __init__
    self.start_session(capabilities)
  File "/home/pi/.venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 342, in start_session
    response = self.execute(Command.NEW_SESSION, caps)["value"]
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/.venv/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
    self.error_handler.check_response(response)
  File "/home/pi/.venv/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 232, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: exited normally.
  (session not created: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
#0 0x005559c72b50 <unknown>
#1 0x0055597effe4 <unknown>
#2 0x0055598214a8 <unknown>
#3 0x00555981da1c <unknown>
#4 0x005559862014 <unknown>
#5 0x0055598618c0 <unknown>
#6 0x0055598279b0 <unknown>
#7 0x0055598286c8 <unknown>
#8 0x005559c46958 <unknown>
#9 0x005559c49948 <unknown>
#10 0x005559c4952c <unknown>
#11 0x005559c38640 <unknown>
#12 0x005559c49f84 <unknown>
#13 0x005559c230a0 <unknown>
#14 0x005559c62404 <unknown>
#15 0x005559c625ec <unknown>
#16 0x005559c71b44 <unknown>
#17 0x007f90deee8c <unknown>
#18 0x007f90e57b18 <unknown>

일단 버전을 확인하라고 하여 확인해봤으나 문제 없었습니다.

$ /usr/bin/chromium --version
Chromium 130.0.6723.116 built on Debian GNU/Linux 12 (bookworm)
$ /usr/bin/chromedriver --version
ChromeDriver 130.0.6723.116 (6ac35f94ae3d01152cf1946c896b0678e48f8ec4-refs/branch-heads/6723@{#1764})

그 외 크롬 실행 옵션에서 ‘–no-sandbox’ 옵션값을 제일 위로 올려봐라는 등 여러 조치사항을 확인하고 적용해봤지만 문제는 해결되지 않았습니다.

그리고 마지막으로 아래와 같은 조치를 취하고 해결되었습니다.

$ sudo apt-get remove chromium
$ sudo apt-get remove chromium-driver

$ pip install --upgrade selenium
$ sudo apt-get install chromium
$ sudo apt-get install chromium-chromedriver

아래는 제가 크롤링에 사용중인 크롬 옵션값 입니다.

options.add_argument('--no-sandbox')                            # 샌드박스 비활성화
options.add_argument('--headless=new')                          # 헤드레스
options.add_argument('--ignore-certificate-errors')             # 인증서 에러 무시
options.add_argument('--incognito')                             # 익명화
options.add_argument('--lang=ko_KR')                            # 언어 설정
options.add_argument('--window-size=1920x1080')                 # pc용 사이즈
options.add_argument('--start-maximized')                       # 브라우저가 최대화된 상태로 실행
options.add_argument('--disable-gpu')                           # 하드웨어 가속화 끔
options.add_argument('--disable-software-rasterizer')           # 소프트웨어 랜더링 가속화 끔
options.add_argument('--disable-dev-shm-usage')                 # 공유메모리 사용 끔
options.add_argument('--blink-settings=imagesEnabled=false')    # 이미지 로딩 비활성화
options.add_argument('--disable-extensions')                    # 확장 비활성화