Monthly Archives: January 2016

Bulk Upload ISOs to vCloud Air

I’ve been leveraging vCloud Air recently for development and testing. The ability to run nested ESXi in the cloud, coupled with the promotions for free credits makes it a convenient playground for running a plethora of applications.

Uploading to vCloud Director has always been cumbersome. Due to the required browser plug-ins, you needed the right combination of versions and your settings had to be just right. But recently Chrome and Firefox have closed the door on old Netscape API that fully broke the ISO upload feature.

VMWare provides a command line tool called OVFTool that allows uploading ISO files (and OVF as well). William Lam over on vGhetto wrote an excellent shell script to wrap OVFTool to make it a little easier. However, I’m still too lazy for that. What I wanted is to upload a bulk of ISO files to my catalog, so I could quickly build environment from scratch.

So, entered vCloudAir_BulkUploadISO.sh, this bash script for Mac will do just that. Set a couple of variables up front; your vCloud Air information and the folder you want to upload. The script will prompt for your credentials and loop through all the ISO files in the folder upload them to your catalog. Clone it from my GitHub page.

Open in your favorite editor, adjust the variables and upload en masse.

By | January 21st, 2016|VMWare|0 Comments

ProcessControl

I was looking through some old code and found a little gem I built almost ten years ago called ProcessControl. I’m sure you’re familiar with the ability to adjust the affinity and priority of processes in Windows. If you’re not, give it a try in Task Manager.

I’ve used these controls quite a bit over the years for numerous purposes. Such as troubleshooting performance by adjusting process priority. Or getting older applications or games to work better by constraining them to a single core (this was common early in the multi-core days before thread management was widespread). Writing the code in my spare time to test an idea I had to improve the performance of a COTS product called SolarWinds Orion, which is an excellent monitoring tool for networking (among other things).

Orion runs multiple components on a single server (web server, business logic, SNMP trap server, NetFlow collector and SNMP poller), however at the time, the SNMP poller was single-threaded; causing a conflict of resources. The other portions of the application that spread their workload across all cores would be on the same core that the single-threaded daemon was assigned automatically by Windows. This contention would slow down the SNMP poller due to the inefficiency.

After testing tuning the affinity of all processes, when the SNMP poller received a dedicated core it showed a vast improvement in the polls per second. However manually adjusting these settings in Task Manager after every reboot or process change is not a viable solution. So in entered a very simple piece a code that would take an XML configuration file with the affinity and priority desire of each process and adjust them programmatically. Running this as a Windows service set to automatically start after boot, and recheck the process settings periodically made it ready for operations.

With the existence of this tool, I found myself using it frequently to solve odd performance situations. Additional benefits were found, such as decreasing the occurrence of processes moving between cores/sockets and the low-level cache rebuilding. Or in processor constrained systems where upgrading hardware wasn’t possible, removing the offending process from core 0 and reducing the process priority would solve stability issues (hardware level interrupts and core operating system processes use core 0, so often OS stability is due to user processes contending with kernel processes).

My most common use of this code was my personal development workstations. When installing lots of SQL, IIS, MySql, PostGres, Apache and more all on the same instance where I do coding; the resource contention between all these applications slows down the GUI. ProcessControl can reduce the cores and priority of all those server daemons, which do not need robust resources to simply test code, or look at configuration details. This, in turn, leaves more resources for the application with a human interface, speeding up the experience.

I’ve posted the code and working binaries on my GitHub page. If you’re tired of manually changing your process priority and affinity or have been looking for a way to tune applications that are having conflicts due to thread management; feel free to use the tool and contribute to the code. It’s not highly complex code, but it gets the job done.

ProcessControl Quick Start

Installation

  1. Create a new folder called “ProcessControl” under “C:\Program Files (x86)\”
  2. Download the entire zipped repository from GitHub (or clone it if you want)
  3. Open the zip file and copy all the contents from “ProcessControl\bin\Release\” into “C:\Program Files (x86)\ProcessControl\”
  4. Navigate to “C:\Program Files (x86)\ProcessControl\” and double-click “ProcessControl.exe”
    1. This will not actually launch the service, but trying to execute the application will ensure you have the needed .Net framework
  5. Use the provided tool “srvinstw” to register ProcessControl.exe as a Windows service
    1. Run as default, so it has access to the processes.
    2. Choose Auto or Manual based on your needs.
  6. Adjust the XML config file “ProcessControlParams.xml” as desired, refer to configuration details below
  7. Start the server

Configuration

There are two configuration files:

ProcessControl.xml is the main application configuration. This has settings for the location of the ProcessControlParams.xml (if you want to host the service and associated files in another location outside of Program Files). As well a setting for the interval to recheck process attributes, by default this is every 15 minutes.

The second XML file, ProcessControlParams.xml, is the meat of the application. The first configuration line with the process name ‘Default’ will adjust the affinity of ALL processes. This is a baseline reset which allows you to clear off a core. The priority control does NOT work for Default. The next line, copied and pasted as many times as you need; adjusts the process of your choosing. You can adjust the priority and/or affinity of (almost) any process (there are a few system processes you cannot control). Here is a quick look at the XML:

<Process Name=“Default” Priority=“” Affinity=“”/>
<Process Name=“MyProcess” Priority=“” Affinity=“”/>
The options for Priority and Affinity?

Priority – these are the standard options:

  • RealTime
  • High
  • AboveNormal
  • Normal
  • BelowNormal
  • Idle

Affinity – this is a little more tricky and controlled by a number to represent all the different configuration options. Below are documented the common options I’ve used in up to an 8 core environment. The options are also documented in the ProcessControlParams.xml file when you download. If you want a combination that is not documented and don’t want to do the math; simply manually set a process to the desired state before starting the service. Step one in launching it to log the status of all existing processes. On the off change you discover more, please update the file on GitHub?

All 8 = 255
4-7 = 240
2,3 = 12
0,1 = 3
4,5 = 48
6,7 = 192
1,2 = 3

Logging

ProcessControl will log into the Windows Application event log. Successful process changes, as well as errors and full exception catch output will all be put into event entries. If you’re having problems with the service, it’s a good bet the information will be in the application log.

By | January 13th, 2016|Code, Tuning|0 Comments