Multithreading – why only 32 threads run when calling futures?

When I found the futures business test:

user=> (time (doall (map deref (for [i (range 1000)]
  #_=>    (future (Thread/sleep 1000))))))
"Elapsed time: 32058.208 msecs"

When the power of 2 is visible, an alarm rings on my head It smells like only 32 threads started

Some target experiments:

user=> (time (doall (map deref (for [i (range 32)]
  #_=> (future (Thread/sleep 1000))))))
"Elapsed time: 1002.302 msecs"
(nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil)
user=> (time (doall (map deref (for [i (range 64)]
  #_=> (future (Thread/sleep 1000))))))
"Elapsed time: 2004.24 msecs"
(nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil)
user=> (time (doall (map deref (for [i (range 65)]
  #_=> (future (Thread/sleep 1000))))))
"Elapsed time: 3005.279 msecs"
(nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil     nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil nil)

Confirm this For each segment of the next 32 missions, let's see it again

Making future codes is

(defmacro future
  "Takes a body of expressions and yields a future object that will
  invoke the body in another thread,and will cache the result and
  return it on all subsequent calls to deref/@. If the computation has
  not yet finished,calls to deref/@ will block,unless the variant of
  deref with timeout is used. See also - realized?."
  {:added "1.1"}
  [& body] `(future-call (^{:once true} fn* [] ~@body)))

The interesting thing about future calls is

fut (.submit clojure.lang.Agent/soloExecutor ^Callable f)]

It is converted to:

volatile public static ExecutorService soloExecutor = Executors.newCachedThreadPool(
    createThreadFactory("clojure-agent-send-off-pool-%d",sendOffThreadPoolCounter));

So this does create an unlimited thread pool

So why only 32 threads run instead of creating 1000 threads and returning one second?

Solution

this

(time (doall (map deref (for [i (range 65)] (future (Thread/sleep 1000))))))
; "Elapsed time: 3005.244983 msecs"

In fact, it should take 65 seconds, because you ask to exit futures on the main thread in turn These futures have not even started because they are lazy The reason you get a multiple of 32 is the blocking behavior of the range

Compare with non blocking version

(time (doall (map deref (for [i (apply list (range 65))] (future (Thread/sleep 1000))))))
; "Elapsed time: 64997.260808 msecs"

As R ö Rd pointed out in his comments, adding another method to eliminate the laziness of future creation solves this problem

(time (doall (map deref (doall (for [i (range 65)] (future (Thread/sleep 1000)))))))
; "Elapsed time: 999.263631 msecs"

Another way to test the completion of all futures is to wait for a promise that will not be delivered before the previous future completion

(defn test-futures [n]
  (let [a (atom 0)
        p (promise)
        f (fn [] (swap! a inc) (when (= @a n) (deliver p n)))]
    (doseq [i (range n)] (future (do (Thread/sleep 1000) (f))))
    (deref p)))

Now you can see that the completion of 64, 65 or 1000 of these futures occurs in about one second

(time (test-futures 64))
; "Elapsed time: 996.262272 msecs"
; 64
(time (test-futures 65))
; "Elapsed time: 996.554436 msecs"
; 65
(time (test-futures 1000))
; "Elapsed time: 1000.247374 msecs"
; 1000