1. 程式人生 > 實用技巧 >queue for thread of Python

queue for thread of Python

queue

https://docs.python.org/3.7/library/queue.html#queue.Queue

支援多生產者和多消費者。

為執行緒通訊設計。

實現三種類型的佇列:

(1)FIFO

(2)LIFO

(3)優先佇列

還有一個簡單佇列,是FIFO的一種易用性特例。

The queue module implements multi-producer, multi-consumer queues.

It is especially useful in threaded programming when information must be exchanged safely between multiple threads.

The Queue class in this module implements all the required locking semantics.

It depends on the availability of thread support in Python; see the threading module.

The module implements three types of queue, which differ only in the order in which the entries are retrieved.

In a FIFO queue, the first tasks added are the first retrieved. In a LIFO

queue, the most recently added entry is the first retrieved (operating like a stack).

With a priority queue, the entries are kept sorted (using the heapq module) and the lowest valued entry is retrieved first.

Internally, those three types of queues use locks to temporarily block competing threads; however, they are not designed to handle reentrancy within a thread.

In addition, the module implements a “simple” FIFO queue type, SimpleQueue, whose specific implementation provides additional guarantees in exchange for the smaller functionality.

優先佇列

https://pymotw.com/3/queue/index.html#priority-queue

Sometimes the processing order of the items in a queue needs to be based on characteristics of those items, rather than just the order they are created or added to the queue. For example, print jobs from the payroll department may take precedence over a code listing that a developer wants to print. PriorityQueue uses the sort order of the contents of the queue to decide which item to retrieve.

#queue_priority.py

import functools
import queue
import threading


@functools.total_ordering
class Job:

    def __init__(self, priority, description):
        self.priority = priority
        self.description = description
        print('New job:', description)
        return

    def __eq__(self, other):
        try:
            return self.priority == other.priority
        except AttributeError:
            return NotImplemented

    def __lt__(self, other):
        try:
            return self.priority < other.priority
        except AttributeError:
            return NotImplemented


q = queue.PriorityQueue()

q.put(Job(3, 'Mid-level job'))
q.put(Job(10, 'Low-level job'))
q.put(Job(1, 'Important job'))


def process_job(q):
    while True:
        next_job = q.get()
        print('Processing job:', next_job.description)
        q.task_done()


workers = [
    threading.Thread(target=process_job, args=(q,)),
    threading.Thread(target=process_job, args=(q,)),
]
for w in workers:
    w.setDaemon(True)
    w.start()

q.join()

functools.total_ordering -- 類全序工具

給類定義全序關係。

讓所有此類的例項都可以進行比較。

https://docs.python.org/3.7/library/functools.html#functools.total_ordering

Given a class defining one or more rich comparison ordering methods, this class decorator supplies the rest. This simplifies the effort involved in specifying all of the possible rich comparison operations:

The class must define one of __lt__(), __le__(), __gt__(), or __ge__(). In addition, the class should supply an __eq__() method.

@total_ordering
class Student:
    def _is_valid_operand(self, other):
        return (hasattr(other, "lastname") and
                hasattr(other, "firstname"))
    def __eq__(self, other):
        if not self._is_valid_operand(other):
            return NotImplemented
        return ((self.lastname.lower(), self.firstname.lower()) ==
                (other.lastname.lower(), other.firstname.lower()))
    def __lt__(self, other):
        if not self._is_valid_operand(other):
            return NotImplemented
        return ((self.lastname.lower(), self.firstname.lower()) <
                (other.lastname.lower(), other.firstname.lower()))

https://www.geeksforgeeks.org/python-functools-total_ordering/

er_none

edit

play_arrow

brightness_4
from functools import total_ordering 
  
@total_ordering
class num: 
      
    def __init__(self, value): 
        self.value = value 
          
    def __lt__(self, other): 
        return self.value < other.value 
        
    def __eq__(self, other): 
          
        # Changing the functionality 
        # of equality operator 
        return self.value != other.value 
          
# Driver code 
print(num(2) < num(3)) 
print(num(2) > num(3)) 
print(num(3) == num(3)) 
print(num(3) == num(5)) 

Total order -- 全序定義

在偏序的基礎上,新增任意元素的可比較性。

https://en.wikipedia.org/wiki/Total_order

For each (non-strict) total order ≤ there is an associated asymmetric (hence irreflexive) transitive semiconnex relation <, called a strict total order or strict semiconnex order,[2] which can be defined in two equivalent ways:

  • a < b if ab and ab
  • a < b if not ba (i.e., < is the inverse of the complement of ≤)

Properties:

  • The relation is transitive: a < b and b < c implies a < c.
  • The relation is trichotomous: exactly one of a < b, b < a and a = b is true.
  • The relation is a strict weak order, where the associated equivalence is equality.

We can work the other way and start by choosing < as a transitive trichotomous binary relation; then a total order ≤ can be defined in two equivalent ways:

  • ab if a < b or a = b
  • ab if not b < a

Two more associated orders are the complements ≥ and >, completing the quadruple {<, >, ≤, ≥}.

We can define or explain the way a set is totally ordered by any of these four relations; the notation implies whether we are talking about the non-strict or the strict total order.

https://mathworld.wolfram.com/PartialOrder.html

https://www.wolframalpha.com/input/?i=totally+ordered+set&assumption=%22ClashPrefs%22+-%3E+%7B%22MathWorld%22%2C+%22TotallyOrderedSet%22%7D

A total order (or "totally ordered set, " or "linearly ordered set") is a set plus a relation on the set (called a total order) that satisfies the conditions for a partial order plus an additional condition known as the comparability condition. A relation <= is a total order on a set S ("<= totally orders S") if the following properties hold.
1. Reflexivity: a<=a for all a element S.
2. Antisymmetry: a<=b and b<=a implies a = b.
3. Transitivity: a<=b and b<=c implies a<=c.
4. Comparability (trichotomy law): For any a, b element S, either a<=b or b<=a.
The first three are the axioms of a partial order, while addition of the trichotomy law defines a total order.
Every finite totally ordered set is well ordered. Any two totally ordered sets with k elements (for k a nonnegative integer) are order isomorphic, and therefore have the same order type (which is also an ordinal number).

FIFO -- Queue

https://github.com/fanqingsong/code_snippet/blob/master/python/thread_queue/queque.py

import threading
import queue


def do_work(item):
    print('now doing work')
    print(item)


def worker():
    while True:
        item = q.get()
        if item is None:
            break
        do_work(item)
        q.task_done()

q = queue.Queue()

num_worker_threads = 3


threads = []
for i in range(num_worker_threads):
    t = threading.Thread(target=worker)
    t.start()
    threads.append(t)


for item in range(3):
    q.put(item)

# block until all tasks are done
q.join()

# stop workers
for i in range(num_worker_threads):
    q.put(None)


for t in threads:
    t.join()