go - Why are goroutines with network i/o being blocked? -

i'm using go 1.1 devel on ubuntu 13.04

go version devel +ebe8bca920ad wed may 15 15:34:47 2013 +1000 linux/386 

according http://golang.org/doc/faq#goroutines

when coroutine blocks, such calling blocking system call, run-time automatically moves other coroutines on same operating system thread different, runnable thread won't blocked.

i'm trying write downloader can download large file in chunks using goroutines , best goroutine i've come with:

func download(uri string, chunks chan int, offset int, file *os.file) {     current := range chunks {          fmt.println("downloading range: ", current, "-", current+offset)          client := &http.client{}         req, _ := http.newrequest("get", uri, nil)         req.header.set("range: ", fmt.sprintf("bytes=%d-%d", current, current+offset))         resp, err := client.do(req)         if err != nil {             panic(err)         }         defer resp.body.close()         body, err := ioutil.readall(resp.body)         if err != nil {             panic(err)         }         file.write(body)     } } 

the full script available @ https://github.com/tuxcanfly/godown/blob/master/godown.go

even though, files being dowloaded , saved correctly, can see second chunk starts when first finishes.

shouldn't chunked downloads run in parallel, or there i'm doing wrong?

you have 1 goroutine downloading chunks.

line 64:

go download(*download_url, chunks, offset, file) 

what want is:

for := 0; < *threads; i++ {     go download(*download_url, chunks, offset, file) } 

this download *threads chunks @ once.

---

after have concurrency working, notice line 29 doesn't work how intend. if chunk 1 finishes before chunk 2, parts written out of order. may want instead use http://golang.org/pkg/os/#file.writeat.

---

you have 2 problems range header.

1. you don't download remainder. if file size 3002 , have 3 threads, request 0-1000, 1000-2000, 2000-3000 , last 2 bytes never downloaded.
2. byte ranges inclusive. means (as can see in previous example) downloading bytes twice. byte 1000 , 2000 requested twice. of course, long write correct locations, shouldn't have of problem.

number 2 easy enough fix changing line 19 from

req.header.set("range: ", fmt.sprintf("bytes=%d-%d", current, current+offset)) 

to this

req.header.set("range: ", fmt.sprintf("bytes=%d-%d", current, current+offset-1)) 

for more information on range header, suggest reading section 14.35 in rfc2616