current position:Home>Python batch get gitlab project code

Python batch get gitlab project code

2022-08-06 18:32:25As for why

More and more companies are now using gitlab to manage their code.There is a problem with gitlab, the free version does not support global code search, which is very troublesome.If you clone all the code to the local, you can easily perform various searches.

Thoughts

gitlab provides api to get projecct list, then you can traverse this list to do git clone

See: https://docs.gitlab.com/ee/api/projects.html#list-all-projects

Generate token

A private token needs to be generated to use the Gitlab API

  • Token generation method:
  • Token is only visible once when it is generated, please save it

Script

Note: gitlab's api can only obtain information about 100 projecct at most each time.This script has page turning function and can support more than 100 items.

# -*- coding: UTF-8 -*-# Passed the test in Python 3.0# Need to create a new AccessToken in gitlab and fill in gitlabTokenimport sysif sys.version_info < (3, 0):import urllibelse:from urllib.request import urlopenimport jsonimport subprocess, shleximport timeimport osgitlabAddr = '201.1.2.115:8099' #git addressgitlabToken = 'Nzyg92-123456aa4ay__y' #gitlab's token, there is a production temporary token in the gitlab settingsfor index in range(10):url = "http://%s/api/v4/projects?private_token=%s&per_page=100&page=%d&order_by=name" % (gitlabAddr, gitlabToken, index)print(url)if sys.version_info < (3, 0):allProjects = urllib.urlopen(url)else:allProjects = urlopen(url)allProjectsDict = json.loads(allProjects.read().decode(encoding='UTF-8'))if len(allProjectsDict) == 0:breakfor thisProject in allProjectsDict:try:thisProjectURL = thisProject['ssh_url_to_repo']thisProjectPath = thisProject['path_with_namespace']print(thisProjectURL + ' ' + thisProjectPath)if os.path.exists(thisProjectPath):command = shlex.split('git -C "%s" pull' % (thisProjectPath))else:command = shlex.split('git clone %s %s' % (thisProjectURL, thisProjectPath))resultCode = subprocess.Popen(command)time.sleep(0.5)except Exception as e:print("Error on %s: %s" % (thisProjectURL, e.strerror))

References

copyright notice
author[As for why],Please bring the original link to reprint, thank you.
https://en.cdmana.com/2022/218/202208061814417824.html

Random recommended