Your IP : 216.73.216.52
a
ؒd�D � @ s& d dl mZ d dlZd dlZd dlZd dlZd dlZd dlZd dlZd dl Z d dl
Z
d dl Z d dl m
Z
mZmZmZmZ d dlmZ d dlmZ d dlmZ d dlmZmZmZm Z m Z m Z m Z d dl m!Z! d d l"m#Z#m$Z$ d
Z%e%e ks�J �d Z&e&e%ks�J �d
Z'e(ed ��oe(ed
�Z)dddd�dd�Z*dd�dd�Z+dddd�dd �Z,ddd d �d d �Z-G d!d"� d"�Z.G d#d$� d$e
�Z/e�r�e0Z1e2ed% e/f e2ed& e1f B Z3G d'd(� d(e4�Z5G d)d*� d*e j6�Z7d+d+d,�d-d.�Z8G d/d0� d0�Z9d1d2d3�d4d5�Z:G d6d7� d7ej;�Z<d8d9d:�d;d<�Z=d=d�d>d?�Z>dS )@� )� annotationsN)�
NamedTuple�NoReturn�Literal�Any�
TYPE_CHECKING)�support)� Namespace)�Regrtest)�runtest� is_failed�
TestResult� Interrupted�Timeout�
ChildError�PROGRESS_MIN_TIME)� setup_tests)�format_duration�
print_warningg >@g �r@�setsid�killpgr
r �bool)�result�ns�returnc C s& t | t�rdS |jr"t| |�r"dS dS )NTF)�
isinstancer Zfailfastr )r r � r �1/usr/lib/python3.9/test/libregrtest/runtest_mp.py� must_stop) s
r ztuple[Namespace, str]�r c C s$ t �| �\}}tf i |��}||fS �N)�json�loadsr )� worker_args�ns_dict� test_namer r r r �parse_worker_args1 s r& �strzsubprocess.Popen)�testnamer r c C sx t |�}|| f}t�|�}tjgt�� �d�d�d�d�|�}i }trNd|d< tj |ftj
tj
dt j dktj
d�|��S ) Nz-uz-mz
test.regrtestz
--worker-argsTZstart_new_session�nt)�stdout�stderrZuniversal_newlinesZ close_fds�cwd)�varsr! �dumps�sys�
executabler Zargs_from_interpreter_flags�USE_PROCESS_GROUP�
subprocess�Popen�PIPE�os�nameZSAVEDCWD)r( r r$ r# �cmd�kwr r r �run_test_in_subprocess7 s4
�������r9 r )r r% r c C s< t | � t| |�}t� ttj|td�dd� t�d� d S )N)�clsT��flushr )r r �printr! r. �EncodeTestResultr/ �exit)r r% r r r r �run_tests_workerP s
r@ c @ s0 e Zd ZdZdd� Zdd� Zdd� Zdd � Zd
S ) �MultiprocessIteratorz8A thread-safe iterator over tests for multiprocess mode.c C s t �� | _|| _d S r )� threadingZLock�lock�
tests_iter)�selfrD r r r �__init__a s
z MultiprocessIterator.__init__c C s | S r r �rE r r r �__iter__e s z MultiprocessIterator.__iter__c C sB | j �( | jd u rt�t| j�W d � S 1 s40 Y d S r )rC rD �
StopIteration�nextrG r r r �__next__h s
z MultiprocessIterator.__next__c C s0 | j � d | _W d � n1 s"0 Y d S r )rC rD rG r r r �stopn s zMultiprocessIterator.stopN)�__name__�
__module__� __qualname__�__doc__rF rH rK rL r r r r rA ] s
rA c @ s. e Zd ZU ded< ded< ded< ded< dS )�MultiprocessResultr
r r' r* r+ � error_msgN)rM rN rO �__annotations__r r r r rQ s s
rQ FTc @ s e Zd ZdS )�
ExitThreadN)rM rN rO r r r r rT s rT c s� e Zd Zdddd�� fdd� Zdd�d d
�Zdd�d d �Zdd�d
d�Zd$ddddd�dd�Zddd�dd�Zddd�dd�Z dd�d d �Z
dd�d d �Z d dd!�d"d#�Z � Z
S )%�TestWorkerProcess�intz'MultiprocessTestRunner'�None)� worker_id�runnerr c sZ t � �� || _|j| _|j| _|j| _|j| _|j| _d | _ d | _
d | _ d| _ d| _
d S )NF)�superrF rX �pending�outputr �worker_timeout�timeout�regrtest�current_test_name�
start_time�_popen�_killed�_stopped)rE rX rY �� __class__r r rF � s
zTestWorkerProcess.__init__r' r c C s� d| j � �g}| �� r"|�d� n
|�d� | j}|rF|�d|� �� | j}|d ur�t�� | j }|�d| jj � �dt
|�� �f� dd� |� S ) NzTestWorkerProcess #�runningZstoppedztest=zpid=ztime=z<%s>� ) rX �is_alive�appendr` rb �time� monotonicra �extend�pidr �join)rE �info�test�popen�dtr r r �__repr__� s
�zTestWorkerProcess.__repr__c
C s� | j }|d u rd S | jr d S d| _tr2| � d�}n| � }td|� �tjdd� z"trft�|jt j
� n|� � W nH t y� Y n8 t
y� } z td|� d|��� W Y d }~n
d }~0 0 d S )NTz process groupzKill ��filer< zFailed to kill z: )rb rc r1 r= r/ r+ r5 r rn �signal�SIGKILL�kill�ProcessLookupError�OSErrorr )rE rr Zwhat�excr r r �_kill� s$ zTestWorkerProcess._killc C s d| _ | �� d S )NT)rd r} rG r r r rL � s zTestWorkerProcess.stop� Nr
rQ )� test_resultr* r+ r c C s t �� | j |_t||||�S r )rk rl ra � duration_secrQ )rE r r* r+ �err_msgr r r �mp_result_error� s z!TestWorkerProcess.mp_result_errorztuple[int, str, str])r% r c C sJ t �� | _|| _z t|| j�}d| _|| _W n d | _� Y n0 z�z�| jrX| � � t
�z(|j | j d�\}}|j
}|d us~J �W nN tjy� | jr�t
�| � � d }d }}Y n. ty� | jr�t
�� Y n0 |�� }|�� }|||fW W | �� d | _d | _S | � � � Y n0 W | �� d | _d | _n| �� d | _d | _0 d S )NF�r^ r~ )rk rl ra r` r9 r rc rb rd r} rT Z communicater^ �
returncoder2 �TimeoutExpiredr{ �strip�rstrip�_wait_completed)rE r% rr r* r+ �retcoder r r � _run_process� sV
�
�z TestWorkerProcess._run_processc
C s� | � |�\}}}|d u r*| �t|�||�S d }|dkr@d| }nb|�d�\}}}|�� }|sbd}n@ztj|td�}W n, ty� } zd| }W Y d }~n
d }~0 0 |d ur�| �t |�|||�S t
||||�S )Nr z Exit code %s�
z Failed to parse worker stdout)Z object_hookz Failed to parse worker JSON: %s) r� r� r �
rpartitionr� r! r" �decode_test_result� Exceptionr rQ ) rE r% r� r* r+ r� �_r r| r r r �_runtest s&
�zTestWorkerProcess._runtestc C s� | j s�zXzt| j�}W n ty. Y W q�Y n0 | �|�}| j�d|f� t|j| j �r\W q�W q t
yt Y q�Y q t y� | j�dt �
� f� Y q�Y q 0 q d S )NFT)rd rJ r[ rI r� r\ Zputr r r rT �
BaseException� traceback�
format_exc)rE r% � mp_resultr r r �run# s
zTestWorkerProcess.runc
C sv | j }|j�� |j�� z|�t� W nH tjtfyp } z*t d| � dt
t�� d|��� W Y d }~n
d }~0 0 d S )NzFailed to wait for z completion (timeout=z): ) rb r* �closer+ �wait� JOIN_TIMEOUTr2 r� r{ r r )rE rr r| r r r r� 6 s
��z!TestWorkerProcess._wait_completed�float)ra r c C sd | � d� | �� sq`t�� | }| j�d| � dt|�� �� |tkr td| � dt|�� �� q`q d S )Ng �?z Waiting for z thread for zFailed to join z in ) ro ri rk rl r_ �logr r� r )rE ra rs r r r � wait_stoppedE s
�z TestWorkerProcess.wait_stopped)r~ r~ N)rM rN rO rF rt r} rL r� r� r� r� r� r� �
__classcell__r r re r rU � s �
;rU zlist[TestWorkerProcess])�workersr c C sN g }| D ]@}|j }|sqt�� |j }|tkrd|t|�f }|�|� q|S )Nz%s (%s))r` rk rl ra r r rj )r� rg �workerr` rs �textr r r � get_running] s r� c @ st e Zd Zddd�dd�Zdd�dd�Zdd�d d
�Zd d�d d
�Zddd�dd�Zddd�dd�Zdd�dd�Z dS )�MultiprocessTestRunnerr
rW �r_ r c C sj || _ | j j| _|j| _t�� | _t| j j�| _| jj d urZt
| jj d | jj d �| _ nd | _ d | _ d S )Ng �?i, )
r_ r� r �queueZQueuer\ rA Ztestsr[ r^ �minr] r� )rE r_ r r r rF k s
�z MultiprocessTestRunner.__init__r c s| � fdd�t d� jjd �D �� _dt� j�� d�}� jjrZ|dt� jj�t� j�f 7 }� �|� � jD ] }|� � qjd S )Nc s g | ]}t |� ��qS r )rU )�.0�indexrG r r �
<listcomp>| s �z8MultiprocessTestRunner.start_workers.<locals>.<listcomp>� z Run tests in parallel using z child processesz" (timeout: %s, worker timeout: %s))
�ranger Zuse_mpr� �lenr^ r r] r� �start)rE �msgr� r rG r �
start_workers{ s
�
��
z$MultiprocessTestRunner.start_workersc C s6 t �� }| jD ] }|�� q| jD ]}|�|� q"d S r )rk rl r� rL r� )rE ra r� r r r � stop_workers� s
z#MultiprocessTestRunner.stop_workerszQueueOutput | Nonec C s� | j jd u}t}tdd� | jD ��r�|r6tjtdd� z| jj |d�W S t
j yZ Y n0 t | j�}|r| j j
s| �dd�|� � qz| jj dd�W S t
j y� Y d S 0 d S ) Nc s s | ]}|� � V qd S r )ri )r� r� r r r � <genexpr>� � z5MultiprocessTestRunner._get_result.<locals>.<genexpr>T)r? r� z running: %s�, r )r r^ �PROGRESS_UPDATE�anyr� � faulthandlerZdump_traceback_later�MAIN_PROCESS_TIMEOUTr\ �getr� ZEmptyr� �pgor� ro )rE Zuse_faulthandlerr^ rg r r r � _get_result� s$ �
z"MultiprocessTestRunner._get_resultrQ )r� r c C s� |j }t|�}|jd ur(|d|j 7 }n$|jtkrL| jjsL|dt|j� 7 }t| j �}|rt| jjst|dd�
|� 7 }| j � | j
|� d S )Nz (%s)z -- running: %sr� )r r' rR r� r r r� r r� r� ro r_ Zdisplay_progress�
test_index)rE r� r r� rg r r r �display_result� s
z%MultiprocessTestRunner.display_result� QueueOutputr )�itemr c C s� |d r"|d }t d|� �� dS | jd7 _|d }| j�|j� | �|� |jrdt|jdd� |jr�| j j
s�t|jt jdd� t |j| j �r�dS dS )z&Returns True if test runner must stop.r r� z regrtest worker thread failed: Tr; ru F)
r r� r_ Zaccumulate_resultr r� r* r= r+ r r� r/ r )rE r� r� r� r r r �_process_result� s
z&MultiprocessTestRunner._process_resultc C s� | � � d| _ztz(| �� }|d u r$q6| �|�}|rq6qW n tyX t� d| j_Y n0 W | jj d urpt
� � | j �
� | �� n(| jj d ur�t
� � | j �
� | �� 0 d S )Nr T)r� r� r� r� �KeyboardInterruptr= r_ Z interruptedr r^ r� Zcancel_dump_traceback_laterr[ rL r� )rE r� rL r r r � run_tests� s(
�
z MultiprocessTestRunner.run_testsN)
rM rN rO rF r� r� r� r� r� r� r r r r r� j s r� r
rW r� c C s t | ��� d S r )r� r� )r_ r r r �run_tests_multiprocess� s r� c s( e Zd ZdZddd�� fdd� Z� ZS )r> z7Encode a TestResult (sub)class object into a JSON dict.r �dict[str, Any])�or c s. t |t�r"t|�}|jj|d< |S t� �|�S )N�__test_result__)r r
r- rf rM rZ �default)rE r� r re r r r� � s
zEncodeTestResult.default)rM rN rO rP r� r� r r re r r> � s r> r� zTestResult | dict[str, Any])�dr c C sB d| vr | S | � d�}t� D ] }|j|kr |f i | �� S q dS )z7Decode a TestResult (sub)class object from a JSON dict.r� N)�pop�get_all_test_result_classesrM )r� �cls_namer: r r r r� � s
r� zset[type[TestResult]]c C sJ d} t h}t|�| krFt|�} g }|D ]}|�|�� � q&|�|� q
|S )Nr )r
r� rm �__subclasses__�update)Z
prev_count�classesZto_addr: r r r r� s r� )?Z
__future__r r� r! r5 r� rw r2 r/ rB rk r� �typingr r r r r rq r Ztest.libregrtest.cmdliner Ztest.libregrtest.mainr
Ztest.libregrtest.runtestr r r
r r r r Ztest.libregrtest.setupr Ztest.libregrtest.utilsr r r� r� r� �hasattrr1 r r& r9 r@ rA rQ r' ZExcStr�tupler� r� rT ZThreadrU r� r� r� Z JSONEncoderr> r� r� r r r r �<module> sR $
[