Category Archives: Python

How to install Ta-Lib and its python library on Ubuntu 22?

Installing TA-lib on an Ubuntu server has its challenges as not only the python library has to be installed but the product should be downloaded and compiled first. Use the following steps to perform the installation:

mkdir -p /app
sudo apt-get install build-essential autoconf libtool pkg-config python3-dev -y
cd /app
sudo wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz
sudo tar -xzf ta-lib-0.4.0-src.tar.gz
cd ta-lib/
sudo ./configure
sudo make
sudo make install
sudo pip3 install --upgrade pip
sudo pip3 install TA-Lib

In case you would be using gitlab pipelines you can use the following job to do the same:

 stages:
   - prepare

 prepare:
   stage: prepare
   script:
     - mkdir -p /app
     - sudo apt-get install build-essential autoconf libtool pkg-config       python3-dev -y
     - cd /app
     - sudo wget http://prdownloads.sourceforge.net/ta-lib/ta-lib-0.4.0-src.tar.gz
     - sudo tar -xzf ta-lib-0.4.0-src.tar.gz
     - cd ta-lib/
     - sudo ./configure
     - sudo make
     - sudo make install
     - sudo pip3 install --upgrade pip
     - sudo pip3 install TA-Lib

Happy trading!

How to remotely list Vmware VMs using python and turn off SSL certificate verification

Since we are automating a number of things in our environment there was a need to list vms remotely using a script. We tried using PHP or the vmware vi perl toolkit but we couldn’t make any of them work.

The problem with that PHP script we tried to use was that the SSL certificate on the vcenter host doesn’t match the hostname and even though we tried everything it was impossible to turn the SSL certificate verification off. We tried to use this script:

https://gist.github.com/scr34m/3490246

The problem with the VI perl toolkit was that the toolkit is kind of old and requires libraries that are not available any longer. It also requires perl version 5.8 so we had to install an older perl version parallel with the current one. After a day spent trying to get it working we gave it up.

The last option was to use the  VMware vSphere API Python Bindings and the pyvmomi-community-samples.  We did the following:

Determine the python version:

Python 2.7.12 (default, Nov 19 2016, 06:48:10)
 [GCC 5.4.0 20160609] on linux2
 Type "help", "copyright", "credits" or "license" for more information.
 >>> import sys
 >>> print (sys.version)
 2.7.12 (default, Nov 19 2016, 06:48:10)
 [GCC 5.4.0 20160609]
 >>>

Since Python 2.7 doesn’t have PIP ( Pip Installs Packages ) installed by default we had to get that downloaded and installed first.

Download get-pip.py from here
python get-pip.py

This installs PIP. Now we need to download and install the vmware python extensions.

pip install pyvmomi

We also need to install git if it is not already installed

apt-get install git

Now install the community samples part. The libraries will be installed into the directory you issue this command from.

git clone https://github.com/vmware/pyvmomi-community-samples.git

Locate and run getallvms.py to list all vms from your vcenter. It is very important that you run this script from the community samples location otherwise it will fail with the following error:

root@zoltan-VirtualBox-kk:/app/images/IPM/python# python getallvms.py
 Traceback (most recent call last):
 File "getallvms.py", line 27, in <module>
 import tools.cli as cli
 ImportError: No module named cli

Run the command using the following syntax:

python getallvms.py -s [vcentername]  -u [vmwreuser] -p [vmwarepassword]

If your hostname doesn’t match the certificate you will be getting the error below. If it matches you should be seeing the list of vms now.

Traceback (most recent call last):
 File "getallvms.py", line 104, in <module>
 main()
 File "getallvms.py", line 78, in main
 port=int(args.port),
 File "build/bdist.linux-x86_64/egg/pyVim/connect.py", line 836, in SmartConnect
 File "build/bdist.linux-x86_64/egg/pyVim/connect.py", line 718, in __FindSupportedVersion
 File "build/bdist.linux-x86_64/egg/pyVim/connect.py", line 638, in __GetServiceVersionDescription
 File "build/bdist.linux-x86_64/egg/pyVim/connect.py", line 604, in __GetElementTree
 File "/usr/lib/python2.7/httplib.py", line 1057, in request
 self._send_request(method, url, body, headers)
 File "/usr/lib/python2.7/httplib.py", line 1097, in _send_request
 self.endheaders(body)
 File "/usr/lib/python2.7/httplib.py", line 1053, in endheaders
 self._send_output(message_body)
 File "/usr/lib/python2.7/httplib.py", line 897, in _send_output
 self.send(msg)
 File "/usr/lib/python2.7/httplib.py", line 859, in send
 self.connect()
 File "/usr/lib/python2.7/httplib.py", line 1278, in connect
 server_hostname=server_hostname)
 File "/usr/lib/python2.7/ssl.py", line 353, in wrap_socket
 _context=self)
 File "/usr/lib/python2.7/ssl.py", line 601, in __init__
 self.do_handshake()
 File "/usr/lib/python2.7/ssl.py", line 830, in do_handshake
 self._sslobj.do_handshake()
 ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

There is no way to switch certificate verification off using a command line parameter so the code has to be changed from:

#!/usr/bin/env python
# VMware vSphere Python SDK
# Copyright (c) 2008-2013 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""
Python program for listing the vms on an ESX / vCenter host
"""

import atexit

from pyVim import connect
from pyVmomi import vmodl
from pyVmomi import vim

import tools.cli as cli


def print_vm_info(virtual_machine):
    """
    Print information for a particular virtual machine or recurse into a
    folder with depth protection
    """
    summary = virtual_machine.summary
    print("Name       : ", summary.config.name)
    print("Template   : ", summary.config.template)
    print("Path       : ", summary.config.vmPathName)
    print("Guest      : ", summary.config.guestFullName)
    print("Instance UUID : ", summary.config.instanceUuid)
    print("Bios UUID     : ", summary.config.uuid)
    annotation = summary.config.annotation
    if annotation:
        print("Annotation : ", annotation)
    print("State      : ", summary.runtime.powerState)
    if summary.guest is not None:
        ip_address = summary.guest.ipAddress
        tools_version = summary.guest.toolsStatus
        if tools_version is not None:
            print("VMware-tools: ", tools_version)
        else:
            print("Vmware-tools: None")
        if ip_address:
            print("IP         : ", ip_address)
        else:
            print("IP         : None")
    if summary.runtime.question is not None:
        print("Question  : ", summary.runtime.question.text)
    print("")


def main():
    """
    Simple command-line program for listing the virtual machines on a system.
    """

    args = cli.get_args()

    try:
        service_instance = connect.SmartConnect(host=args.host,
                                                user=args.user,
                                                pwd=args.password,
                                                port=int(args.port))

        atexit.register(connect.Disconnect, service_instance)

        content = service_instance.RetrieveContent()

        container = content.rootFolder  # starting point to look into
        viewType = [vim.VirtualMachine]  # object types to look for
        recursive = True  # whether we should look into it recursively
        containerView = content.viewManager.CreateContainerView(
            container, viewType, recursive)

        children = containerView.view
        for child in children:
            print_vm_info(child)

    except vmodl.MethodFault as error:
        print("Caught vmodl fault : " + error.msg)
        return -1

    return 0

# Start program
if __name__ == "__main__":
    main()

…to the code below. Changes are highlighted with bold ( might not be easy to see at first )

#!/usr/bin/env python
# VMware vSphere Python SDK
# Copyright (c) 2008-2013 VMware, Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""
Python program for listing the vms on an ESX / vCenter host
"""

import atexit
import ssl

from pyVim import connect
from pyVmomi import vmodl
from pyVmomi import vim

import tools.cli as cli


def print_vm_info(virtual_machine):
    """
    Print information for a particular virtual machine or recurse into a
    folder with depth protection
    """
    summary = virtual_machine.summary
    print("Name       : ", summary.config.name)
    print("Template   : ", summary.config.template)
    print("Path       : ", summary.config.vmPathName)
    print("Guest      : ", summary.config.guestFullName)
    print("Instance UUID : ", summary.config.instanceUuid)
    print("Bios UUID     : ", summary.config.uuid)
    annotation = summary.config.annotation
    if annotation:
        print("Annotation : ", annotation)
    print("State      : ", summary.runtime.powerState)
    if summary.guest is not None:
        ip_address = summary.guest.ipAddress
        tools_version = summary.guest.toolsStatus
        if tools_version is not None:
            print("VMware-tools: ", tools_version)
        else:
            print("Vmware-tools: None")
        if ip_address:
            print("IP         : ", ip_address)
        else:
            print("IP         : None")
    if summary.runtime.question is not None:
        print("Question  : ", summary.runtime.question.text)
    print("")


def main():
    """
    Simple command-line program for listing the virtual machines on a system.
    """

    args = cli.get_args()

    try:

        context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
        context.verify_mode = ssl.CERT_NONE

        service_instance = connect.SmartConnect(host=args.host,
                                                user=args.user,
                                                pwd=args.password,
                                                port=int(args.port),
                                                sslContext=context
)

        atexit.register(connect.Disconnect, service_instance)

        content = service_instance.RetrieveContent()

        container = content.rootFolder  # starting point to look into
        viewType = [vim.VirtualMachine]  # object types to look for
        recursive = True  # whether we should look into it recursively
        containerView = content.viewManager.CreateContainerView(
            container, viewType, recursive)

        children = containerView.view
        for child in children:
            print_vm_info(child)

    except vmodl.MethodFault as error:
        print("Caught vmodl fault : " + error.msg)
        return -1

    return 0

# Start program
if __name__ == "__main__":
    main()

This solution was based on code displayed in this discussion:

https://github.com/vmware/pyvmomi/issues/235