db46 Checksum Error During MacPorts Upgrade

Problem

When upgrading my MacPorts installation, using sudo port upgrade outdated, I ran into the following error:

--->  Verifying checksum(s) for db46
Error: Checksum (md5) mismatch for patch.4.6.21.1
Error: Checksum (md5) mismatch for patch.4.6.21.2
Error: Checksum (md5) mismatch for patch.4.6.21.3
Error: Checksum (md5) mismatch for patch.4.6.21.4
***
The non-matching file appears to be HTML. See this page for possible reasons
for the checksum mismatch:

***
Error: Target org.macports.checksum returned: Unable to verify file checksums
Log for db46 is at: /opt/local/var/macports/logs/_opt_local_var_macports_sources_rsync.macports.org_release_ports_databases_db46/main.log
Error: Problem while installing db46

Solution

sudo port clean --all db46

Subsequently, running sudo port upgrade outdated worked as expected.

Posted in OS X, Technology | 1 Comment

Mac OS + MySQL-python-1.2.3: ImportError | Library not loaded | libmysqlclient.16.dylib

Attempting to get Python communicating with MySQL via MySQLdb on OS X resulted in the following error when I tried to import MySQLdb:

Traceback (most recent call last):
  File "./dbtest.py", line 3, in 
    import MySQLdb
  File "build/bdist.macosx-10.6-universal/egg/MySQLdb/__init__.py", line 19, in 
  File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 7, in 
  File "build/bdist.macosx-10.6-universal/egg/_mysql.py", line 6, in __bootstrap__
ImportError: dlopen(/Users/fitzsimj/.python-eggs/MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so, 2): Library not loaded: libmysqlclient.16.dylib
  Referenced from: /Users/fitzsimj/.python-eggs/MySQL_python-1.2.3-py2.6-macosx-10.6-universal.egg-tmp/_mysql.so
  Reason: image not found

I eventually found a solution mentioned here. I added the following line to /etc/bashrc:

export DYLD_LIBRARY_PATH=/usr/local/mysql/lib/

There is another solution suggested, using install_name_tool. That approach involves Ruby, which I’m not using.

Posted in Technology | 7 Comments

BluetoothAPIs.h Broken in Windows SDK

Summary

The Microsoft Windows SDK versions 7.0 and 7.1 appear to have broken BluetoothAPIs.h header files.

Update:  Microsoft has fixed the errors in the Bluetooth header as of SDK version 8.0 for Windows 8.

Details

So far, I have uncovered two types of errors in this header file:

    • The use of

#pragma deprecate

      instead of

#pragma deprecated

      , causing compiler warnings.

    • Several callback function pointer type definitions omit the

CALLBACK

      (

__stdcall

    ) calling convention, causing a crash.

The first error simply results in compiler warnings.

warning C4068: unknown pragma

The second type of error results in dereferencing of an invalid memory location when using BluetoothRegisterForAuthenticationEx and BluetoothAuthenticateDeviceEx. This is because the standard calling convention (__cdecl) assumes that the caller will clean up the stack. Since the caller in this case is assuming that the callback function minded its own stack, it immediately pops ESI, placing zero into the register:

5EBCFFE2  mov         ecx,dword ptr [ebp-4]  
5EBCFFE5  pop         esi  
5EBCFFE6  xor         ecx,ebp  
5EBCFFE8  pop         ebx  
5EBCFFE9  call        @__security_check_cookie@4 (5EBDBBBBh)

Later, ntdll.dll dereferences memory at ESI + 4, triggering an access violation:

774A8301  test        byte ptr [esi+4],4

“Unhandled exception at 0x774a8301 (ntdll.dll) in [Application]: 0xC0000005: Access violation reading location 0x00000004.

Solution

To the compiler warnings, I replaced all instances of

#pragma deprecate

with

#pragma deprecated

To fix the crash bug, I added the CALLBACK calling convention keyword to PFN_AUTHENTICATION_CALLBACK and PFN_AUTHENTICATION_CALLBACK_EX. They now appear as follows:

typedef BOOL (CALLBACK *PFN_AUTHENTICATION_CALLBACK)(LPVOID pvParam, PBLUETOOTH_DEVICE_INFO pDevice);

typedef BOOL (CALLBACK *PFN_AUTHENTICATION_CALLBACK_EX)(__in_opt LPVOID pvParam, __in PBLUETOOTH_AUTHENTICATION_CALLBACK_PARAMS pAuthCallbackParams);

Interestingly, the function pointer type definitions for the attribute-enumeration and device-selection callbacks (PFN_BLUETOOTH_ENUM_ATTRIBUTES_CALLBACK and PFN_DEVICE_CALLBACK, respectively) are defined correctly, using CALLBACK or WINAPI. I suspect that the inconsistency is because someone at Microsoft was using the /Gz compiler switch, making __stdcall the default calling convention.

Posted in Technology, Win32, Windows | 2 Comments

Alternate Data Streams (Metadata) on Files in NTFS

Introduction

Alternate Data Streams (ADS) allow arbitrary metadata to be associated with files and directories on Windows NTFS. Alternate data streams are the Windows implementation of forks. The apparent size of the file will be unchanged, and most applications and users are unaware of their existence. If a file is moved, any alternate data stream will move along with it, as long as the destination is on an NTFS drive.

The command line can access alternate data streams using redirection operators. Streams are specified on the command line as filename:stream name.

Creating an Alternate Data Stream

As an example, a string is written into an ADS named hidden, which is associated with file test.txt:

C:\test>echo Hidden text > test.txt:hidden

The file appears to be empty, though as detailed below, the metadata is intact and associated with the file:

C:\test>dir test.txt

06/24/2010  01:33 PM                 0 test.txt

Viewing an Alternate Data Stream

The metadata can be viewed by redirecting from it to more:

C:\test>more < test.txt:hidden
Hidden text

The name and content of the ADS can be anything (see 'Details' below for restrictions):

C:\test>echo Arbitrary string > test.txt:arbitraryName

C:\test>more < test.txt:arbitraryName
Arbitrary string

Listing Files With Alternate Data Streams

On Windows Vista and later, a list of alternate data streams can be obtained using DIR /R:

C:\test>dir test.txt /R

06/24/2010  01:33 PM                 0 test.txt
                                    38 test.txt:arbitraryName:$DATA
                                    28 test.txt:hidden:$DATA

On earlier operating systems, the SysInternals utility Streams can be used:

C:\test>c:\tools\SysInternals\streams.exe test.txt

Streams v1.56 - Enumerate alternate NTFS data streams
Copyright (C) 1999-2007 Mark Russinovich
Sysinternals - www.sysinternals.com

C:\test\test.txt:
   :arbitraryName:$DATA 38
          :hidden:$DATA 28

Alternate Data Streams on Directories

Metadata can be added to directories the same way it's added to files:

C:\test>mkdir test2

C:\test>echo ADS on a directory > test2:someText

C:\test>dir /r

06/25/2010  11:27 PM    <DIR>          .
06/25/2010  11:27 PM    <DIR>          ..
06/25/2010  11:27 PM    <DIR>          test2
                                    42 test2:someText:$DATA

C:\test>more < test2:someText
ADS on a directory

Details

Stream Naming

To be more accurate, streams are specified as filename:stream name:stream type. It appears that the only stream type accessible from the command line is $DATA, which is why it's optional. All of the stream types are listed in the WIN32_STREAM_ID structure documentation. The default data stream is unnamed, so filename::$DATA will contain the file's data:

C:\test>echo This is the file > file.txt

C:\test>echo This is the stream > file.txt:stream

C:\test>more < file.txt::$DATA
This is the file

C:\test>more < file.txt:stream:$DATA
This is the stream

Stream names are generally held to the same requirements as any filename. One interesting difference is that stream names can contain characters whose integer representations are in the range from 1 through 31. Refer to Naming Files, Paths, and Namespaces (MSDN) for details.

Note that when using streams with files having a single letter name, the filename should be prefixed with a period and backslash. The reason for this is Windows drive names. For example, does "echo hello > c:test" refer to a stream named test on file c, or does it refer to a file test on drive c?

Executing Streams

As of Windows Vista, it is no longer possible to execute directly from an alternate data stream. On Windows XP and earlier, the Start command was used, similar to start somefile.ext:hiddenExecutable.

Editing with Notepad

Notepad can be used to create and edit alternate data streams. The File Open dialog doesn't recognize stream syntax, however, so the file must be created and opened using command line parameters. Notepad will insist on appending .txt to the stream name.

Programmatic Access

Microsoft provides a sample program in C++, demonstrating how to open and write to an alternate data stream.

Real-World Applications

Downloaded Executables

Since Windows XP SP2, when a file is downloaded from the Internet and executed (assuming a zone-aware browser), this warning is displayed:

Windows displays this warning because the web browser tagged the executable with a alternate data stream named Zone.Identifier:

C:\test>dir /r setup.exe

06/25/2010  12:10 PM           680,467 setup.exe
                                    26 setup.exe:Zone.Identifier:$DATA

By redirecting this stream to more, we can see its contents:

C:\test>more < setup.exe:Zone.Identifier
[ZoneTransfer]
ZoneId=3

The PowerShell blog has more information on zone identifiers.

Viruses

The W2K.Stream virus used alternate data streams.

Additional Resources

Posted in C++, Scripting, Technology, Windows | 1 Comment

VLC File and Network Cache Settings

Inevitably, by the time VLC upgrades itself and wipes its settings, I’ve forgotten where the file and network buffering settings are.

  1. Go to the VLC preferences (Tools | Preferences).
  2. Select “All” under Show settings, or “Show Advanced” in the lower, left corner of the dialog.
  3. In the tree view on the left, select Input/Codecs | Access Modules | File.

Default values are 300ms for local files, with an extra 900ms for network access. Doubling each value gave me smooth playback again.

Posted in Technology | Leave a comment

Converting Yodlee’s CSV Financial Data to OFX

I was attempting to use csv2ofx to convert Yodlee MoneyCenter’s exported comma separated value data into an OFX file, for use in GnuCash. I repeatedly got an error when exporting:

KeyError: 'Split Type'

This turned out to be due to two bad lines written to the top of each file. Removing those led to additional errors, “KeyError: 'Account Name'” and “KeyError: 'Transaction Id'“.

In the end, I had to load the CSV file into a spreadsheet and perform the following steps:

  1. Delete a few lines at the top, which list the account name.
  2. Add column “Account Name”, populate it with the name of the account.
  3. Add column “Transaction Id”, populate it with an incrementing list of numbers.
Posted in Finance | Leave a comment

Console and Cygwin Don’t Show All Files on 64-bit Windows

Running Cygwin bash or Console under 64-bit versions of Windows, directory listings sometimes differ from results shown in Explorer or cmd.exe. For example, Defrag.exe is visible to the default command shell (CMD):

C:\Windows\System32>dir Defr*

07/13/2009  05:39 PM           183,296 Defrag.exe
07/13/2009  05:40 PM            16,384 defragproxy.dll
07/13/2009  05:40 PM           291,328 defragsvc.dll

The same listing under Cygwin bash yields no results:

$ cd /cygdrive/c/Windows/System32

$ ls Defra*
ls: cannot access Defra*: No such file or directory

In fact, on this system, Cygwin reports sees only 2,465 files in System32, while CMD sees 2,857 files.

This is because Cygwin bash and Console are 32-bit applications, and I’m running 64-bit Windows. With filesystem virtualization on Windows, when a 32-bit process attempts to access %SYSTEMROOT%\System32, it is redirected to %SYSTEMROOT%\SYSWOW64. Ironically named, System32 contains 64-bit applications, while SYSWOW64 contains 32-bit applications.

Posted in Technology, Windows | 1 Comment

Add PNG Support to WordPress Plugin “Image Shadow”

The Image Shadow plugin for WordPress works pretty well. Unfortunately, it only adds shadows to JPEG images. PNG images can be added very easily, but they will be converted to JPEG format in the image cache. This isn’t optimal for things like screenshots, but the images still look good enough for my needs.

Edit wp-content/plugins/image-shadow/image-shadow.php. Find this line:

if (!preg_match('/\.(jpeg|jpg|jpe)$/i', $src)) return $src;

Add |png to the match expression:

if (!preg_match('/\.(jpeg|jpg|jpe|png)$/i', $src)) return $src;
Posted in Technology, Wordpress | Leave a comment

WordPress Twentyten Theme Without Header Image

The latest version of the Twentyten theme seems to have introduced a bug in the header. If you opt for no header image, you’ll end up with a large space and a broken image icon in your header:

The problem is a conditional in wp-content/themes/twentyten/header.php which writes out an img tag even if get_header_image() returns an empty string:

   // Check if this is a post or page, if it has a thumbnail, and if it's a big one
   if ( is_singular() &&
      has_post_thumbnail( $post->ID ) &&
         ( /* $src, $width, $height */ $image = wp_get_attachment_image_src( get_post_thumbnail_id( $post->ID ), 'post-thumbnail' ) ) &&
         $image[1] >= HEADER_IMAGE_WIDTH ) :
      // Houston, we have a new header image!
      echo get_the_post_thumbnail( $post->ID, 'post-thumbnail' );
   else : ?>
      <img src="<?php header_image(); ?>" width="<?php echo HEADER_IMAGE_WIDTH; ?>" height="<?php echo HEADER_IMAGE_HEIGHT; ?>" alt="" />
   <?php endif; ?>

Replace the else with an elseif which checks to see if the header image string is empty:

   elseif ( get_header_image() != "") : ?>
Posted in Technology, Wordpress | 1 Comment

Install Windows 7 from USB Thumb Drive

The Windows 7 USB/DVD Download Tool can copy a Windows .ISO file to a USB thumb/pen drive.

Posted in Technology, Windows | Leave a comment