syntax - Parsing Python function calls to get argument positions -

i want code can analyze function call this:

whatever(foo, baz(), 'puppet', 24+2, meow=3, *meowargs, **meowargs) 

and return positions of each , every argument, in case foo, baz(), 'puppet', 24+2, meow=3, *meowargs, **meowargs.

i tried using _ast module, , seems thing job, unfortunately there problems. example, in argument baz() function call itself, couldn't find simple way length. (and if found one, don't want bunch of special cases every different kind of argument.)

i looked @ tokenize module couldn't see how use arguments.

any idea how solve this?

this code uses combination of ast (to find initial argument offsets) , regular expressions (to identify boundaries of arguments):

import ast import re  def collect_offsets(call_string):     def _abs_offset(lineno, col_offset):         current_lineno = 0         total = 0         line in call_string.splitlines():             current_lineno += 1             if current_lineno == lineno:                 return col_offset + total             total += len(line)     # parse call_string ast     call = ast.parse(call_string).body[0].value     # collect offsets provided ast     offsets = []     arg in call.args:         = arg         while isinstance(a, ast.binop):             = a.left         offsets.append(_abs_offset(a.lineno, a.col_offset))     kw in call.keywords:         offsets.append(_abs_offset(kw.value.lineno, kw.value.col_offset))     if call.starargs:         offsets.append(_abs_offset(call.starargs.lineno, call.starargs.col_offset))     if call.kwargs:         offsets.append(_abs_offset(call.kwargs.lineno, call.kwargs.col_offset))     offsets.append(len(call_string))     return offsets  def argpos(call_string):     def _find_start(prev_end, offset):         s = call_string[prev_end:offset]         m = re.search('(\(|,)(\s*)(.*?)$', s)         return prev_end + m.regs[3][0]     def _find_end(start, next_offset):         s = call_string[start:next_offset]         m = re.search('(\s*)$', s[:max(s.rfind(','), s.rfind(')'))])         return start + m.start()      offsets = collect_offsets(call_string)         result = []     # previous end     end = 0     # given offsets = [9, 14, 21, ...],     # zip(offsets, offsets[1:]) returns [(9, 14), (14, 21), ...]     offset, next_offset in zip(offsets, offsets[1:]):         #print 'i:', offset, next_offset         start = _find_start(end, offset)         end = _find_end(start, next_offset)         #print 'r:', start, end         result.append((start, end))     return result  if __name__ == '__main__':     try:         while true:             call_string = raw_input()             positions = argpos(call_string)             p in positions:                 print ' ' * p[0] + '^' + ((' ' * (p[1] - p[0] - 2) + '^') if p[1] - p[0] > 1 else '')             print positions     except eoferror, keyboardinterrupt:         pass 

output:

whatever(foo, baz(), 'puppet', 24+2, meow=3, *meowargs, **meowargs)          ^ ^               ^   ^                      ^      ^                                ^  ^                                      ^    ^                                              ^       ^                                                         ^        ^ [(9, 12), (14, 19), (21, 29), (31, 35), (37, 43), (45, 54), (56, 66)] f(1, len(document_text) - 1 - position)   ^      ^                               ^ [(2, 3), (5, 38)]